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FOREWORD 


Tills report is prepared in two volumes. Volume I reports the Executive 
Summary and the findings of the research. Volume n contains the appendices 
to the Rnal report. The appendices list detailed documentation which supports 
the research findings. This includes specific materials and procedures 
used in: a) the open and closed forms of the knowledge tests, b) the full 
mission simulations, and c) the paper and pencil tests. 
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APPENDIX A 


DOD Interest In The CIFE Problem 


I. Literature: mostly event or problem specific articles (e.g. , engine fail- 
ures, electrical system failures, thunderstorm penetration, wind shear, 
low visibility approaches, etc. ) but some studies contalnli^ generalisa- 
tions on workload, perception, distraction, simulation, stress measure- 
ment, etc. 

A. Indices (and Repositories*) 

1. Air University Index of Military Periodicals 
(Air University Library*) 

2. Defense Documentation Center (DDC)* 

3. National Technical Information Services (NTIS)* 

4. Transportation Research Information Services (THIS) 

B. Selected Journals 

1. Avt'.tlon, Space, and Environmental Medicine 

2. Aerospace Safety 

3. MAC Flyer 

4. TAC Attack 

5. Combat Crew 

6. Interceptor 

7. Soviet Military Review 

8. Journal of Human Stress 

9. U.S. Army Aviation Digest 

10. Approach (U.S. Navy) 

11. TIG (Tlie Inspector General) Brief 

n. Examples of Recent/ Cur rent Research 

A. Air Force Fll^t Dynamics Laboratory - mlsslmi management 
techniques for large aircraft OpUot workload, stress, and decision 
making in relation to automatic fll^t control systems) 

B. Aerospace Medical Research Laboratory - pilot workload assessment; 
measures of human operator performance in operational stress environ- 
ments 

C. AF Humsn Resources Laboratory - evaluation of an emergency 
procedures training program 
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D. AT Ottioe of Solonttfio Reoearoh - divided attention and tank 
workload In conm>l failura dataotUm and daclalon making 

E. Savoy Aviation Raaaarob Laboratory (Univ. of lUinoia) - 
anhancamant of hun&an affactivanasa in ayatam daaign, training, 
and operation 

F. Percaptronica, Inc. - amergancy prooaduraa training packagaa 

O. AF School of Aaroapaca Madicina - atraaa factora in aviation 

m. Air Force Accident/liKSidant Data Baaaa (available to ua but raatrictad 

*Tor Official Uae Only") 

A . Haaardoua Air Traffic Raporta - narrative of aocidanta that did 
not hai^n (aimilar to ASRS) 

B. Main Frame Computer File - lengthy deacription of what hai^nad 
and findinga of the accident invaatigation board, 1962 to preaant 

C. Management information Technical Syatem (MITS) (the "one liner 
liner report") aircraft data, phaae of fllid^t, time, weather, (atalltiea, 
fire damage, parta failure, cauae codea (up to two), ejection data, 
accountability code, and brief remarka, 1962 to inreaent. 

D. Accident Inveatigation Reporta - a complete report on each accident 
including who, what, where, when, how, and aometimea, why; 
includea peraonal teatimony if available 

IV. Current Air Studiea 

A. "Change Pace" > by AF Safety Center, lo<^ at cauaea for increaae 

in accident rate during 1976-78 time frame, triea to anawer the "why" 
queation aa well aa aummariae ^at happened 

B. Project Haaty Blue - by Air Training Command and the AF Human 
Reaoureea Laboratory, deala widi aelection criteria for candidatea 
to enter U8AF pilot training 

V. Additional Methodologiea for CIFE Reaearoh (not previoualy diacuaaed) 

A. Heuriatic programming of a robot - ref. '^eationnaire Theory: 
Modelling of the Pilota ' :ental Load" by Dominique Soulatgea, Office 
National d'Etudea et de T^'icherchea Aeroapatlalea, Parla 

B. Deriving weif^ta for a aet of activitiea according to aeveral criteria 
via a pairwiae compariaon matrix and anal 3 rzing a deciaion maker's 
cona latency - ref. A Scaling Method for Prior itlea ir ITierarchical 
Structurea by Thomaa L. Saaty 
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APPENDIX B 


Annotated Bibltograpliy 


<l) The Meticmal Tranaportatlon Safety B^rd’e "Annual Review of Aircraft Accident 
Dau" for U.8. General Aviation, Calendar Year 1977, was reviewed to 
verify that the literature aearched ia pertinent. Fatal and non-fatal accident 
data including cauaea/factora cited by accident type were reviewed. It waa 
eaaily verified that pilot Jut^ent waa die uncterlying element in a large 
percentage of accidenta. What ia not preaent are the reaulta of near-accidenta 
and train of thought prior to any accident. 

(2) Billinga, Charlea E. , Ralph J. Gerke, and Robert L. Wick. Jr. , "Compariaona 

of Pilot Performance in Simulated and Actual Fllfd^t" in Clinical Medicine , 
March, 1975, performed in-flight and aimulator ejq^rimenta with experi- 
enced pilota. Under varying doaea of aecobarbital, the aubjecta flew multiple 
IIB approachea. The data were more atrongly aaaociated with the drug level 
in die aimulator than in die airplane. The related effecta were more 
cons latent in the aimulator. Improvement in performance auggeative of learning 
eff‘^rta were aeen in the aimulator but iK>t in actual fli^t. 

The most important coiwlusion was the the GAT-1 simulator ia a uaeful and 
senaitive device for atudiea of the effecta of mild streaa on pilot performance, 
but extrapolation of aimulator data to die flight environment moat be approached 
with conalderable caution. 

(3) Bola, Eric H. and Janice E. Eiaele, "General Aviation IFR Operational Problema", 

in NASA Contractor Report 159022, April, 1979, have preaented general 
aviation IFR (^rational problema aa more of an overview which deflnea 
problema than aa a ayatem anaiyaia. Signlf leant ia the diacuaoion of cockpit 
workload and atreaa. (Alao aignlficant the lack of footnotiiy lor direct and 
derived data: We are left to aaaume the data are properly treated. ) Thia 
paper ia an effort to review all die fac^ora influencing the general aviation 
pilot in the world of IFR and etmaidera future developmenta currently in 
planning. It ia an intereating point to atart eonaiderlng critical in-fli^t 
eventa. 
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(4) Brugglnk, Gerald M., "Managing Emergencies", In The MAC Flyer , April, 

1980. In his discussion of managing emergencies, speaks of problems 
of crew coordination In crew-served aircraft. Ha cites decision-making 
In a true emergency situation as an area that cannot be simulated. He 
claims that relying on the pat procedures in an established checklist Is a 
gateway to a dead end road when a "non-standard" emergency develops. 

He identifies smoke, fire, takeoff and landing problems as real emergencies 
where Immediate action must be taken. CXher problems, at altitude, allow 
more time for analysis and correction, this article is basically a call for 
aviation safety through operating Intelligence and conscience; It is not a 
technical treatise. 

(5) Butterbaug^, Larry C. , "Crew Workload - Technology Review and Problem 

Assessment", In AF Flight Dynamics Laboratory Technical Memorandum 
AFFDL-TM-78-74-FGR, has reviewed and discussed the applications and 
resulting technical reopiirements of pilot/crew workload measurement/ 
prediction methods. 

In addition, existing pilot/crew workload measurement/prediction technology 
is Identified and reviewed. An assessment of the adequacy of these tech- 
niques relative to the identified requirements Is performed. Generally, the 
state of workload measurement/prediction technology Is not sufficiently 
developed for measuring/predicting total mission-derived workload, but is 
sufficient for part- task applications. 


(6) Cavalli, Daniel, "Discrete-Time Pilot Model". The objective of this paper 
was to demonstrate the originality of his approach widi regards to already 
existing pilot models and to present recently obtained results. He considers 
the pilot's behavior as a discrete-time process where the decision making 
has a sequential nature. This model contrasts very clearly with previous 
approaches, namely the quasi- linear model which follows from classical 
control theory and the optimal control model which considers the human 
operator as a Kalman estimator-predictor. He also considers that the 
pilot's objective may not be adequately formulated as a quadratic cost 
functional to be minimized, but rather as a more fuzzy measure of the close- 
ness with which the aircraft follows a reference trajectory. 

All model parameters, in the digital program simulating the pilot's behavior, 
have been successfully compared in terms of standard- deviation and performance 
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with those of professional pilots in IFR configuration. The first practical 
application of the pilot model has been the study of its performance 
degradation when the aircraft model static margin decreases. 

Of significance in his model of the human sensor /decision/ action loops 
is his observation that there is only a single loop in operation at a given 
time. It is noted that this is a most fundamental difference between a 
human pilot and an autopilot. This may be a valuable consideration when 
designing various decision making scenarios. 

(7) Ceausu, Valerlu, "The Decision In the Flight Activity", Revue Raumainte 

Des Sciences Soc tales -Serle De Psychologic, Vol. 15, #2, 1971, 

Ceausu makes an Involved, text-like analysis of pilot decision making. 

His major input to the CIFE analysis is not just the various laboratory 
models he develops but is the concept of present time versus future time. 
The tests used present time without anticipatory capabilities available to 
the subject. He acknowledges the constraining effect on his subject. It is 
suggested that even the principles of laboratory psychological investigation 
be revised to accomodate the "temporal dimensions". 

(8) Connor, T. M. and Hamilton, C. W. , "Final Report on Evaluation of Safet>’ 

Programs With Respect to the Causes of Air Carrier Accidents" to the FA A, 
May 16, 1979. Connor and Hamilton cite, among other things, a need to 
emphasize efforts at system level programs aimed at optimizing operator 
performance in specified temporal and spatial conditions. New program 
initiatives are required that address human error problems in behavioral 
terms at detailed cause/factor levels. This will provide a very broad 
knowledge base upon which further research, programs, and data can grow. 


(9) Honeywell Systems and Research Center, "Crew Workload Assessment - 
Development of a Measure of Operator Workload", Technical Report 
•AFFDL-TR-78, June 1977 -September 1978. Honeywell developed a 
quantitative measure of operator information processing workload for use 
in crew-station evaluation. A conceptual relationship between task performance. 
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task difficulty, and operator workload was formulated which predicts a 
positive correlation between performance and workload over an inter- 
mediate range of task difficulties. 

After statistical evaluation, it was found that the forearm electromyogram 
amplitude, respiration amplitude, and respiration duration metrics have 
ordinal scale characteristics which can be used to compare dec Igr options 
relatively. But further analysis is needed to generalize this or a related 
metric to workload estimation in real-world flight tasks. 

(10) Curry, RenwickE., John K. Lauber and Charles E. Billings, 'Experiments 

In Pilot Decision-Making During Simulated Low Visibility Approaches". 

Curry, et. al. , have devised a way to simulate stress in the laboratory 
with significant correlation to pilot decision making during low visibility 
approaches. They observe that despite a vast accumulation of operational 
experience with the conduct of low visibility instrument approaches, little 
is understood about the decision-making behavior of pilots who fly these 
approaches. Likewise, there is little information regarding the man, 
system, and task-related factors which Influence this decision-making 
behavior. Such information is essential for the rational design of new 
systems, or for the redesign of existing systems in order to correct 
known deficiencies. 

They have assumed that it is necessary to use a simulation task which 
incorporates both kinds of variables. Informational and psjc hological, 
to successfully study pilot decision-making behavior in the laboratory. 

Their paper describes the preliminary experiments in the measurement 
of decisions and the inducement of stress in simulated low visibility approaches. 

(11) Damos, Diane L. , "Residual Attention as a Predictor of Pilot Performance", 

Human Factors, 1978. Damos discusses an experiment where sixteen student 
pilots performed a task combination designed to measure residual attention. 
Scores on this combination were correlated with performances on fli^t checks 
administered periodically during flight training. The multiple correlation 
between performances on the flight checks and the task combination increased 
as the students progressed through flight training. The usefulness of residual 
attention as a predictor of pilot performance is dlscrassed. 

It is suggested that future research in this area should be concerned primarily 
with determining the long-range predictive validity of measures of residual 
attention. The paper does not address judgment training. 
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(12) DeMaio, Joseph, Stanley R. Parkinson, and John V. Crosby, "A Reaction 
Time Analysis of Instrument Scanning", Human Factors, 1978. 

DeMaio, et. al. , have made a study of Instrument scanning reaction time. 
Detection and latency performance of Instructor pilots and students pilots 
was compared in a task which required searching an aircraft instrument 
display for target deviations from a desired course. Performance of 
Instructors was superior to that of students on both detection and latency 
measures. The use of stable fixation queues was reflected in the latency 
performance of students. The latency performance of Instructors was much 
less affected by the use of such queues . Results were interpreted to 
suggest that experienced pilots are able to place a greater reliance on 
peripheral vision in scanning instrument displays. 


(13) Dieterly, Duncan L. , "Problem Solving and Decision Making: An Integration", 
NASA-Ames Research Center Report, AFHRL-TR-78, May, 1978. 

Dieterly reviewed the areas of problem soPung and decision-making to 
determine if a central approach could be identified. The approach would 
then be applied to training others in those areas . However , in the face of 
no model standardization, synthesis of research proved ineffective. The 
basic structure developed in Dieterly’s research may be the basis upon 
which this structure could be developed but that was not within the scope 
of his project. He submits that a training program embodying such concepts 
would be an effective tool in decision/problem resolution training. 

(14) Dieterly, Duncan L., "Accident Analysis: Application of the Decision/ Problem 
State Analysis Methodology", NASA-.Ames Research Center Report, 
AFHRL-TR-78, August, 1978. In a subsequent paper Dieterh developed 
and applied a methodology for analyzing the decision/problem state. This 
methodology' was developed to improve the present capability to explain the 
causes of human error accidents. 

He assumed that a successful outcome may be obtained only through the 
management of the decision/problem state. The decision/problem state 
is the set of decision/problem conditions that must be resolved to obtain 
an outcome. In the example he provides, the element of state management 
is not apparent in the available record. The approach he suggested provides 
a more systematic and comprehensive method for studying one major aspect 
of human error in accidents. It is another step in mode standardization. 
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(15) Boeing Aerospace Company, WAM User's Manual , September 30, 1976 through 
June 30, 1977, Report #0180-20247-3. Boeing has published, under 
contract, a document which is the User's Manual for a computer program 
called WAM (Workload Assessment Model). The manual contains a technical 
description of the analytical functions of the model; instructions for the 
preparation of input data; instructions for the retrieval of WAM outputs; a 
description of the deck structure and control cards required for WAM 
execution; and a description of diagnostic messages generated by WAM. 

WAM is used by human engineering analysts to study crew workloading at 
a task or subtask level. Estimates of workload, for a given temporal 
interval, are based upon the time available versus the time required to per- 
form all tasks within the interval. WAM pro^idss both printed and plotted 
workload data for each operator as a function of several workload channels 
(i.e., vision, hands, feet, cognitive, auditory, verbal). A task shifting 
option is provided which automatically shifts tasks to reduce workload when 
overload conditions are encountered. 

WAM is one of several computer models developed under the Computer 
Aided Function-Allocation and Evaluation System (CAFES) Program. WAM 
is written in Fortran IV for use on the CDC 6600 computer with the 
KRONOS 2. 1 Operating System and the RUN Compiler. 


(16) FAA General Aviation News , June, 1978, "Don't Give Up The Ship". 

In the FAA General Aviation News two separate critical in-flight events 
(in a twin Beech and aLearjet)were discussed. They happened to the 
same 19,000 hour pilot. He summarized both events by saying, "‘ite 
one thing I'm sure of is that the experience shock me up enough to nakt 
me realize something important: none of us pilots is so calm and co< l 
and collected but that in a moment of Intense crisis we nre liable to start 
worrying and stop searching for the answer that may be right under our 
nose. In my case (for the second event) the answer was a little to the right 
and below my nose, but it was there all the time". 

But it was there all the time . . . The editor noted the key point: "This 
pilot's confidence in his training enabled him to solve a critical in-flight 
problem". Here, then, are the two problem areas pertinent to this report, 
the quality of training tempered by experience, and confidence in some. 
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(17) Forsyth, Donna L. and John D. Shaughnessy, 'Single Pilot IFR Operating 

Problems Determined From Accident Data Analysis", NASA Technical 
Memorandum, September, 1978. Forsyth and Shaughnessy made an 
examination of Single Pilot Instrument Flight Rule (SPIFR) operations 
from 1964-75 NTSB accident files. They concluded that problem areas 
exist In pilot workload, low visibility at night due to fog and low ceilings, 
Icing on aircr:ift not deicer equipped, Imprecise navigation, failure to 
remain above minimum altitudes, mismanagement of fuel and low Instru- 
ment time. Some suggested areas of research Include new types of 
deicing or anti-icing equipment, standardized navigation Instrument 
displays, improved fuel management systems and better methods for 
pilots to safely acquire experience and increase proficiency in SPIFR 
operations. 

(18) Gartner, Walter B and Miles R. Murphy, "Pilot Workload and Fatigue: 

A Critical Survey of Concepts and Assessment Techniques", a Technical 
Note published by NASA. Gartner and Murphy have an in-depth literature 
review on pilot workload and fatigue. It Is rigorous enough for them to 
be able to draw conclusions on the way the problem should be addressed, 
measured, and managed. They look at measurement of pilot effort, 
cognition, and other studies in sleep research. 

The overall study, then, addresses the principal unresolved Issues in 
conceptualizing and measuring pilot workload and fatigue. These Issues 
are seen as limiting the development of more useful working concepts 
and techniques and their application to systems engineering and management 
activities. A conceptual .analysis of pilot workload and fatigue, an overview 
and critique of approaches to the assessment of these phenomena, and a 
discussion of current trends in the management of unwanted workload and 
fatigue effects are presented. Refinements and innovations in assessment 
methods are recommended for enhancing the practical significance of work- 
load and fatigue studies. 

(19) Kowalsky, Nestor B, Richard L. Masters, Richard B. Stone, Gary L. Babcock, 

and Eugene W. R>q)ka, "An Analysis of Pilot Error-Related Aircraft 
Accidents", NAS.A Contractor Report Final *<CR-2414, June, 1974. 

Kowalsky, et. al., present im in-depth analysis of air carrier accident 
investigation problems, techniques, and solutions. They propose and apply 
a systematic methodology to examine the characteristics of flights prior to 
accidents (environmental factors, alrcrafl systems, pilot performance, 
facilities, policies, procedures). They also include the contingencies per- 
taining at the time of the accident, to afford consideration of all the elements 
Interfacing and interacting In the operation of a complex aeronautical system. 
Of significance is their underlying treatment of decision making. 
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(20) Levit, Robert A., "Human Behavior In Extreme Situations: Generalizations 

From a Review of the Disaster Literature", Proceedings of the Human 
Factors Society 22nd Annual Meeting , 1978. Levit provides a short review 
of disaster literature and some exemplary human factors studies. He 
makes generalizations on principles of disaster management from the 
standpoint of human factors professionals. The items relevant to the 
fli^t regime are from a study by Keating and Loftus on voice alarm 
systems (VAS): (1) The system must emphasize the communication of 
what is happening, why and what to do about it, (2) Messages should be 
unambiguous and communicated in a manner which instills a sense of 
confidence, order and control in the recipients, (3) Messages should oe 
delivered before emergency cues have reached a large segment of the 
populace, (4) All essential information should be repeated twice using 
relatively common words, and (5) The attentional value of emergency 
messages is enhanced if segments are delivered by alternating male and 
female voices. 

(21) Hart, Sandra G. , "A Cognitive Model of Time Perception", presented at the 

56th Annual Meeting of the Western Psychological Association, April, 

1976. Hart defines the terms and basic model used in the research and 
papers described below. 

(22) Hart, Sandra Gall and Duncan McPherson, "Airline Pilot Time Estimation 

During Concurrent Activity Including Simulated Flight", presented at the 
47th Annual Meeting of the Aerospace Medical Association, May, 1976. 

In further developing the utility of pilot time estimation, Hart and 
McPherson clarify the functional relationships beb^een the length and 
variability of time estimates and concurrent task variables. This is one 
way to provide an unobtrusive and minimally loading additional task 
that is sensitive to differences in flying conditions and aircraft instrumenta- 
tion associated with complex piloting tasks. 

(23) Hart, Sandra Gall, Duncan McPherson and Leslie L. Loomis, 'Time Estimation 

As A Secondary Task to Measure Workload: Summary of Research", 
presented at the 15th Annual Manual, 1978. Hart, McPherson, and Loomis 
have outlined the results of a longer series of experiments designed to 
evaluate the utility of time estimation as a secondary measure of piloting 
workload. Actively produced intervals of time were found to increase in 
length and variability, whereas retrospectively produced intervals decreased 
in length although they also Increased in variability with the addition of a 
variety of flight-related tasks. If pilots counted aloud while making a 
production, however, the impact of concurrent activity was minimized, at 
least for the moderately demanding primary tasks that were selected. The 
effects of feedback on estimation accuracy and consistency were greatly 
enhanced if a counting or tapping production technique was used. This 
compares with the minimal effect that feedback had when no overt timekeeping 
technique was used. 
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Actively made verbal estimates of sessions filled with different actlvltes 
decreased In length as the amount and complexity of activities performed 
during the interval were Increased. Retrospectively made verbal 
estimates, however, increased In length as the amount and complexity 
of activities performed during the interval were increased. These 
results support the suggestion that time estimation provides a useful 
index of the workload involved in performing concurrent tasks . 


(24) Jensen, Richard S. , "Pilot Judgment: Training and Evaluation". Jensen 

covers a large area in the state-of-the-art of pilot decision making. 

He examines various techniques and measures of training and testing 
effectiveness, defines judgment in two ways, and considers the problems 
in teaching judgment. 

In looking at learning principles, available training media, computer-aided 
instruction and situational emergency training, Jensen leads into the 
problem of judgment evaluation. Many points are made which could lead 
to scenario development for evaluatii^ and training in situational emergencies. 

(25) Murphy, M. R. , "Analysis of Eighty-four Commercial Aviation Incidents: 

Implications for a Resource Management Approach to Crew Training", in the 
1980 Proceedings Annual Reliability and Maintainability Symposium Manual. 
Murphy considers air crew performance from a resource management view- 
point. He observes that resource management training should be concentrated 
on: (1) Interpersonal communications, with .Air Traffic Control (ATC) infor- 
mation of major concern, (2) Task management, mainly setting priorities 
and appropriately allocating tasks under varying workload levels, (3) Planning, 
coordination, and decision making concerned with preventing and recovering 
from potentially unsafe situations in certain aircraft maneuvers. Problem 
solving and leadership skills were implicated as factors in a sufficient 
number of incidents to require further study. Leadership, social skills, 
and role- issue effects may be under-reported in voluntarily submitted 
incident data: more systematic study is recommended. Some problem 
areas are identified for which design changes are apparently In order, 
particularly the ATC Interface. 


All 


(26) NASA Aviation Safety Reporting System: Ninth Quarterly Report, NASA 

Technical Memorandum 78608. This report publishes the results of 
a study on Distraction - A Human Factor in Air Carrier Hazard Events. 

As a frequent contributor or cause of hazardous events, distraction 
incidents (where attention is diverted from aircraft management, such 
as heading control) can be type-classified. Both air and ground opera- 
tions suffer accidents where the normal crew coordination designed as 
a human-engineered series of tasks is interrupted. By following 
established cockpit priorities, reformatting charts, SIDS, approaches, 
etc. , reading distractions can be reduced. Autopilot monitoring ("who's 
flying the airplane?"), improved alr/ground communications, weather, 
passenger problems, controller inputs, and many other items are con- 
sidered as distractions. Reducing the cockpit workload results in more 
time for vigilance and anticipation of events requiring judgment. The 
underlying theme of the report is that a good train of thought is easily 
and often disturbed throug^i distraction. 

(27) Roscoe, Alan H. , "Stress and Workload in Pilots", Aviation, Space, and 

Environmental Medicine, April, 1978. Roscoe observes that several 
studies have highlighted the increase in physiological activity which 
occurs in pilots during flight and especially during takeoffs and landings. 
For example, it has been clearly demonstrated that pilots' heart rates 
increase during the landing approach to reach a peak at or just before 
touchdown. These changes have been attributed to workload and to 
psychological or emotional stress. This paper examines a number of 
test pilots' heart rate responses recorded during various flight trials 
involving different types of aircraft. Examples Include ramp take- 
offs in a VTOL fighter, automatic landings in fog, supersonic flight 
through monsoon rain, and a sortie in which the pilot developed acute 
appendicitis. It is concluded that heart rate responses in experienced 
pilots are influenced almost entirely by workload-related factors and 
not by emotional stressors, such as risk and anxiety. Because of the 
emotional overtones of the word "stress", It is su^ested that the term 
workload should be used when referring to the reason for increased 
cardiovascular activity in pilots. 

^28) Thorpe, Jack A, Elizabeth L. Martin, Bernell J, Edwards, and Edward E. 

Eddowes, "Situational Emergency Training: F-15 Emergency Procedures 
Training Program", Interim Report for Air Force Systems Command, 

July 1975 - January 1976. Thorpe, et. al., discuss the comparative 
strengths and weaknesses of Situational Emergency Training (SET) versus 
Boldface checklist reactions. Since the latter is simply the critical steps 
that must be taken to reduce an emergency, there is room for pilot error 
in maintaining control of the aircraft, developing another emergency, and 
losing positional awareness. There is little thought (judgment) applied, 
just immediate actions. 
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The objective of their work was to develop and evaluate an emergency 
procedures training program for the F-15. The three applicable in 
all emergency/abnormal situations specified for F-15 operations are: 

<a) maintain aircraft control, (b) analyze the situation and take the proper 
action, and (c) land as soon as practicable. The traditional emergency 
procedures common to other USAF weapons systems featuring Boldface 
procedures which must be committed to memory do not exist for the F-15. 

The strengths and weaknesses of both approaches were noted. Five 
conclusions were derived from this comparative analysis: (a) the trad- 
itional Boldface approach has several deficiencies which may reduce the 
probability that judgment will be exercised when needed, (b) SET is more 
comprehensive than Boldface, encourages the development of judgment, 
and centers training around all three emergency rules listed above, 

(c) the underlying concept of SET is situational training, an approach 
which systematically manipulates the important dimensions of the 
emergency situation. The pilot is taught to discriminate the relevant 
from the irrelevant dimensions of the situation, a discrimination process 
which is fundamental to exercising judgment, (d) pilots report a positive 
attitude towards SET training sessions, which in turn has resulted in 
what supervisors feel is a more productive training program, and (e) by 
using a scenario development procedure, it is hypothesized that SET can 
be more effective. 

(29) Trolllp, Stanley R. , "The Evaluation of a Complex Computer-Based FUght 

Procedures Trainer", Human Factors, 1979. Skills such as flying 
holding patterns are taught in planes or simulators. An alternative method 
is to use computer- assisted Instruction (CAI) which emphasizes training 
requirements rather than physical fidelity. Such a program was written 
and evaluated. Traditional ground school methods were compared with the 
CAI method. All subjects completed a training seqi’ence in a ground 
trainer. Those taught by computer performed better and attained criterion 
quicker with significantly fewer critical errors. Results indicate the CAI 
offers an effective alternate to the costly trainers currently in use. 

(30) Verstynen, Harry A. , "A Possible Role For the Pilot in the Future ATC 

System", presented at the fall conference of the Air Traffic Control 
Association, October, 1978. Verstjmen presents an overview of the 
role of the pilot In the future ATC system. He admits to bias and seeks 
the reader's judgment of validity. He poses some interesting problems 
and possible outcomes. He concludes that the role of the pilot in the future 
ATC system is likely to be substantially changed from the role of the pilot 
today. The two major factors effecting this role will be the shifting of 
certain ATC functions from the ATC system to the pilot and the develops 
ment of onboard processing capabilities which will relieve the pilot of 
many of the functions to which machines are better suited than humans. 


The pilot will progressively become less of a control manipulator 
and more of a manager of a system which combines traditional outer 
loop tasks such as navigation, planning, and resource management 
with new roles as a tactical situation manager. 

(31) Walden, RexS. and William B. Rouse, "Queueing Model of Pilot Decision- 
making In a Multitask Flight Management Situation", IEEE Transactions 
On Systems, Man, and Cybernetics , Vol. SMC-8, December, 1978. 
Walden and Rouse model pilot decision making as a queueing problem. 

Allocation of decisionmaking responsibility between pilot and computer 
is considered, and a flight management task, designed for the study 
of pi lot- computer interaction, is discussed. A queueing theory 
model of pilot decision making in this multitask control and monitoring 
situation is presented. An experimental investigation of pilot decision 
making and the resulting model parameters are discussed. 

They conclude that the queueing formulation of the control and monitoring 
situation presented is attractive. With the exception of the control task 
service rate, the parameters of the model are "eminently measureable", 
at least in controlled experiments . WTtether this Is transferrable to the 
real world is not discussed. 
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APPENDIX C 


Trip Summary Outline - W.C. Giffin, T.H. Rockwell, J. Schofield 


NTSB - March 12, 1979 

A. Participants 

1. James Danaher, Chief Operational Factors Division 

2. Gerrlt Walhout, Chief Human Factors Division 

3. Dave Kelly, Chief Information System Division 

B. Suggested Contacts 

1. Al Diehl, Human Factors Engineering - NTSB 

2. Ward Edv • rds. Prof. (?) (Fighter Pilot Decisions) 

3. Jim I,oon s, Battelle (NTSB Data) 

4. Emil Spleza, Fort Rucker (Viet Nam Data) 

C. Interesting Data 

1. 80 man hours to report BA accident 

2. "In-House" documents behind blue cover reports 

3. Danaher list of resource management accidents for future study 

4. 65 pilot factors In NTSB data 

5. Accident identified by "nearest post office" 

6. Two years cases at NTSB - rest in archives 

7. FA A training approval through air carrier training office (ATCO) 

8. Fort Rucker factor analysis of Viet Nam data 


ATA - March 12, 1979 

A . Participants 

1. H.G. (Grady) Gatlin, Director of Operations 

2. Frank Brady, Director NAS Systems Engineering (ATA) 

3. Larry Gillespie, MGR, DF FLT. OPS. 

4. Vern Ballenger, Director of Engineering 

5. Bob Smith, ATC 

6. Peter Duprey 

B. Suggested Contacts 

1. Airline Safety Directors (see Gatlin letter 3/13/79) 

2. Dave Tliomas, GAMA 



3. Jim Gannet, Boeing Human Factors 

4. Fli^t Safety Foundation 

5. NASA/ Langley (Air Crew Performance Evaluation) 

6. ICAO, (Accident/incident Report for INTN'L) 

7. Captain H.T. Nunn, Northwest (Loft) 

C. Interesting Data 

1. Line oriented fli^t training 

2. ASRS - best hope for incident data 

3. Gatlin 'horror show" of unreported incidents 

4. American Airlines BAC III - record of human factors information 

(24 parameters) 

5. United Airlines - Safety Awareness Program 


3. Mitre Air Transport DIvisfon - March 12, 1979 

A. Participants 

1. Dr. J.S. Matney 

2. Dr. Glen Kenney - ATC Human Factors 

3. Pat McKay - Division Chief 

B. Interesting Data 

1. Gear study to FFS modernization program ? (e.g. , natural for 

computer aid to distressed pilot) 

2. Lovelace report - on site air carrier pilot analysis 

3. ATC system errors - 600/yr. - in-house monitoring 

4. Improve work habits to reduce system errors 

5. Problem with unreported errors 

6. System error causes 

1. Lack of controller awareness of own limitations 
ii. Lack of standard work habits 
lii. Delay action (reluctant to tell A/C what to do) 


4. ALPA - March 13, '.^79 
A. Participants 

1. Bill Edmunds, Human Performance Specialist 

2. Dick Stone, Delta Captain and Chairman of Committee 
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B. Suggested Contacts 

1. Homer Mouden, Flight Safety Foundation (703-820-2777) 

2. Topmiller, WPAFB 

3. J.D. Smith, United 

4. Mack Eastburn, United 

C. Interesting I>ata 

1. Simulator; demonstration of proficiency for FAA only 

2. Ask pilot “what scares you ?" 

3. CIFE - events which take A/C out of normal flight envelope 

4. Crew often unaware of participation In CIFE 

5. False alarms compromise warning systems 

6. Training not consistent with real world 

7. Difference between success and failure Is Information 

8. ALFA gold medal award (annual) 

9. Lovelace report (stone Involved) 

10. United and Allegheny good sources 

11. Competition In cockpit - who can solve problem first 

12. Crew lounge Interviews 

13. Resources management - why give decision maker manipulati 

functions ? 


AGFA - March 13, 1979 

A. Participants 

1. Russel Lawton 

B. Suggested Contacts 

1. John Shaughnessy - NASA/Langley (804-827-3917) 
(time line analysis for single pilot IFR) 

C. Interesting Data 

1. AGFA market survey might include CIFE questions 

2. Plantation party seminar 

3. AGPA has docket on member accident reports 

4. Lawton work'.ng on paper "Pilot Distress" 

5. Royal A/F - pilot workload/stress 
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DSR - Bolling AFB - March 13, 1979 (J. Schofield) 


A. Participants 

1. Jack Thorpe 

B. Suggested Contacts 

1. Dr. Wayne Waag - AFHRL/FT, Williams AFB (8-474-6945) 

(C-5 Pilot Performance Measures Project) 

2. Hans Heinrich and Lt. Col. Rudy Hartzog - SAC/lnstructional 

Systems Development (8-271-2674/3015) 

C. Interesting Data (AF06R Contracts and Grants) 

1. Herbert A. Colle (Wright State University) - a capacity theoretic 

approach to workload assessment 

2. Diane Damos (State University of New York) - training efficient 

multiple-task strategies 

3. Herbert Land and Stuart E. Dreyfus (University of California, 

Berkeley) - formal versus situational models of expert decision- 
making 

4. Daniel Gopher - task load and operator attention capacity in time- 

sharing performance 

5. Gary S. Krahenbuhl (Arizona State University) - stress and learning 

in the flying training environment 

6. Luigi Lucaccini, Amos Freedy, and Rosemarie Hopl-Weichel 

(Perceptronics, Inc.) instructional system development and 
evaluation of situational emergency training 


7. United Airlines - April 19, 1979 

A . Participants 

1. J.D. Smith, Vice President 

2. Tom Dawe, Flight Operations Safety Task Force 

B. Interesting Data 

1. NASA contract with UAL to bring SST into ATC system 

2. UAL involved in basic program for ASRS - foreign and domestic 

3. UAL internal report system 

a) co. immunity 

b) 180 reports 1977-1979 

c) ATC incidents, equipment malfunctions, procedures 

d) all United reports go to ASRS 
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4. monitored approach concept - crew coordination 

5. 24 hr. maintenance function avail to patch for help 

6. Navy study - Dr. Alkov - fatals came from "best” pilots 

who couldn't cope with marital, budget, etc. problems 

7. command training module - how to train decision makers 

8. "Irregularity" reports - UAL has 5000/yr. 

9. cardinal sin - operating by assumption 
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APPENDIX D 


Open Form Knowledge Survey with Answer Key 
and Description of Six Pilot Knowledge Areas 


You have been asked to help the OSU Systems Research 
Group develop a means of testing pilot knowledge of 
aircraft sub- system operation. Please give a short 
answer to each of the following questions in the space 
provided. If you need more space, use the back of 
that page. 

Althou^ this questionaire will not be graded, you 
may wish to compare your responses to our own answer 
key. 

Thank you for your help and cooperation. 
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1. What Is the only method to assure the fuel boost pump Is operating? 


2. Name a few design Innovations intended to minimize "sloshing" in a fuel tank. 


3. Name a source in the cockpit of determining the amount of usable fuel an 
airplane is capable of carrying (other than the Aircraft Operating Manual). 


4. Does a turn coordinator and a turn-and-bank indicator provide you with 
identical information? If no, what is the difference? 


5. Should ammeter indications fluctuate with changes in engine RPM with an 
alternator system? 


6. What would be an indication of an alternator malfunction? 


7. Aircraft operating manuals usually suggest the fuel boost pump be turned on 
for take-offs and when conducting low altitude (less than 1000' AGL) operations 
(on airplanes that require them). What is the reason for this ? 


8. What is the function of a voltage regulator ? 


9. Is structural icing considered a likely event above the clouds ? Why ? 


10. What is the standard adiabatic lapse rate? 


11. What indications would tell the pilot his alternator system Is operating 

normally, or "acceptably"? Do the indications of "normal operation" change 
during the course of a fll^t ? 
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12. Define IAS, CAS, TAS. 


13. If an alternator is removed from the rest of the electrical system by an 
automatic circuit protection device (i.e., an auto-reset circuit breaker), 
how can the pilot attempt to bring the alternator back into service ? 


14. If smoke is noticed in the cabin, what indicators would tell the pilot it is 
"electrical" in origin? What should he do ? 


15. When the battery power is sufficiently "mn down", an "alternator restart" 
may be impossible. What does this mean and why does It happen? 


16. What does an ammeter tell the pilot ? 


17. WTiat does the term "imusable fuel" mean? 


18. What could cause a sharp, sudden decrease in RPM, and pronounced 
engine roughness ? 


19. What range of temperature and atmospheric conditions is most likely to 
to cause carburetor ice ? 


20. What alrsp>eed and altimeter errors are associated with the use of an 
interior alternate static air source in unpressurized airplanes ? 


21. What is "detonation" and when does it occur ? 
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22, If it becomes necessary to continue operation with the use of full carburetor 
heat, what action should be taken to Insure smooth engine operation ? 


23. What does a suction gauge, or instrument air gauge. Indicate? 


24. In what temperature range should carburetor heat not be used? 


25. What Is the reason for not using carburetor heat during ground operations ? 


26, Periodically, an ordinary directional gyro needs to be reset to keep its 
indications consistent with that of the magnetic compass. Why is this so? 


27. What do ground based weather and airborne weather radar systems detect 
and display ? 


28. What are the symptoms indicating the onset of carbon- monoxide poisoning? 


29. Is there any ground-assisted instrument approach procedure, other 

than PAR or ASR, which can be made in an emergency? (Grotmd- assisted 
assumes that all the pilot needs is a two-way radio). 


30. What is the difference between a "slaved" directional gyro and a non-slaved 
one ? 


31. Which gyro flight instrument is most likely to tumble last? 


32. How can a pilot determine that his alternator is operating properly (before 
he takes off)? 
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33. Master switches, on aircraft with alternator systems, usually consist of 
two switches labeled BATTERY and ALTERNATOR which can be turned 
ON or OFF. Which of these two switches, when turned OFF, automatically 
turns off the other side as well ? 


34. Describe the possible ways a pilot could remove the alternator from the 
rest of the electrical field, and operate solely on battery power. 


35. What causes a gj'roscopic flight Instrument to tumble, l.e. , give grossly 
erroneous or nonsense Information? 


36. Name all of the "unsafe gear" Indications you can recall, that a manufacturer 
uses in airplanes with retractable gear. 


37. In an airplane equipped with retractable landing gear, what guidelines 
normally determine the point of retraction ? 


38. Explain your procedure for power application and power reduction in an 
airplane equipped with a constant- speed propeller 


39. What determines the configuration of cowl flaps during a climb ? 


40. Define the terms "service ceiling” and "absolute ceiling”. 


41. Are airspeeds marked on the airspeed indicator in terms of IAS, CAS, or TAS " 


42. If there is a loss of oil pressure, how would a constant -sp>eed propeller be 
affected ^ 
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43. Begin with the slowest "V" speed shown on an airspeed indicator and 
describe each in order, including its' significance. 


44. Some gyro flight instruments are "vacuum” driven. What does this mean? 


45. Where does carburetor ice form (induction Ice)? 


46. How can a pilot usually tell when detonation occurs ? 


47. What would you do if you encountered severe turbulence in IFR conditions ? 


48. Assume you encounter instrument conditions Immediately after lift-off. 

A short time later you notice your airspeed indicator showing a steady 
decrease, altimeter still shows field elevation, and your VSI shows a 
zero rate of change. (Attitude indicator still shows normal climb). What 
is your response ? 


49. What is the cause of most engine failures in flight? 


50. How is vacuum system pressure differential kept at a constant value 
throughout the range of normal operating RPMs and altitudes ? 


51. Suppose your pitot tube became totally and rapidly blocked (i.e. , ice) at 
5000' while you were climbing to 10,000'. Assuming the aircrafts true 
airspeed remained the same during the climb through 5000', what indica- 
tions vould you expect from your airspeed indicator, altimeter, and vertical 
speefi indicator? 
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52. Assuming the failure of all gyroscopic flight Instruments, how can a 
pilot directionally control an airplane In a decent thrcnigh an overcast ? 


53. At what altitude should leaning be attempted under normal cruising power 
(less than 75%), in a direct drive, fixed pitch, float type carburetored 
airplane ? 


54. In what free air temperature range would expected structural Icing? 


55. If you suspect your air filter has become obstructed or Iced-over in flight, 
what action should you take and why? 


56. What is the best method for temporarily correcting prelgnltlon and detonation? 


57. What could cause a gradual drop In RPM and engine roughness (float type 
carburetor) ? 


58. What flight Instruments would be affected by a total blockage of the static 
port ? 
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Answer Key To Open Form Knowledge Survey 


1. Turn on the fuel pump before starting the engine. There should be an ino <:ase 
in fuel pressure as indicated on fuel pressure gauge. 

2. Baffles and Rubber Bladders. 

3. On the Fuel Selector valve or the Fuel Shut-off valve. 

4. No. A turn and bank Indicator reveals direction and rate of turn. A turn 
coordinator provides that information plus the rate and direction of roll. 

Both instruments have "balls” which tell the quality of coordinated flight. 

5. No. 

6. An excessively high rate of charge, any discharge, illumination of an over- 
voltage light, if equipped, or a tripped alternator circuit breaker. 

7 . This is done to insure an uninterrupted flow of fuel to the engine in the event 
the mechanical pump fails. Engine failure at such a critical altitude may 
not allow enough time for a successful restart. 

8. To regulate the rate of charge of the battery, and to place electrical load on 
the alternator. 

9. No, visible moisture is necessary. 

10. 2°C/I000 ft. 

11. A zero rate or a slightly positive rate of charge Indicates a normal operation. 
Yes, normal indications do change during the course of a flight: after engine 
start, charge rate is sli^tly higher than at other times. 

12. IAS = Indicated Air Speed = speed read directly from indicator. 

CAS = Calibrated Air Speed = IAS corrected for position and instrument errors. 
TAS = True Air Speed = calibrated air speed correct for nonstandard tempera- 
ture and pressure. 

13. Turn off alternator switch for a moment, then turn back on. 

14. Smells like burning insulation. Turn off all switches and isolate faulty 
equipment by turning switches on one at a time. 
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15. It occurs because the battery power is Insufficient to close the relay to 
bring the alternator into the electrical field, if this happens it means the 
battery will not charge even if the airplane is jump started. 

16. The rate of battery charge or discharge, or the amount of electrical load 
assumed by the alternator. 

17. Fuel remaining in the tanks which cannot be relied upon for use in all nor- 
mal flight attitudes or conditions. 

18. Magneto malfunction or Internal engine failure. 

19. 32“ to 80“ F with relative humidity 50% or more. 

20. Airspeed reads higher than actual and altimeter reads higher than actual, 
and there is more of a lag in indications. 

21. Detonation is the uncontrolled explosion of fuel in the power stroke caused 
by overheating or improper grades of fuel. 

22. Lean the mixture. 

23. The difference bet\veen pressure inside the system (vacuum network) and 
the pressure outside the system. 

24. Below 15“ F. 

25. Unfiltered air enters the carburetor with the carburetor heat "on". 

26. It is due to precession from motion and drag in the bearings. 

27. Precipitation. 

28. Feelings of sluggishness, tightness across forehead, warmth, headaches, 
throbbing, pressure at the temples. 

29. Yes, an emergency D.F. approach. 

20. A slaved directional gyro continuously corrects itself with respect to a 

remote magnetic compass, whereas a non- slaved one must be reset manually. 

31. A turn-and-bank Indicator or a turn coordinator. 

32. Turn on landing light (or other high load device). For a charge-discharge 
ammeter, needle should flicker and return to original position. For a 
load type ammeter, needle should Indicate higher than originally. 
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33. Battery. 


34. 


35. 


36. 


37. 


38. 


39. 


40. 


41. 


42. 


43. 


44. 


45. 


46. 


47. 


Turn off alternator master switch, or trip the alternator circuit breaker. 

They tumble when they are forced beyond their mechanical limits, or 
when there is an insufficient rate of rotation. 

Gear horn, gear -in- brans it lights, unlit gear down light, 'barber poles", 
gear position indicator, or gear emergency extension handle. 

Retract at point in take-off where insufficient runway ahead remains for 
landing. 

Adding power prop forward then throttle; reducing power, throttle back then 
prop. 

Cylinder head temperature . 

Service ceiling is the altitude at which maximum rate of climb Is 100 fpm. 
Absolute celling is altitude where maximum rate of climb is zero. 

CAS. 

Prop would move to hi^ RPM (low pitch). 

= stall speed - landing configuration. 


Vgj = stall speed - gear and flaps retracted 



= maximum speed with flaps extended 


Vno ~ maximum structured cruising speed 



= never exceed speed 


Air is sucked out of instrument case by a vacuum and inco.nlng air is 
directed at rim of gyro rotor, causing it to spin. 

Throttle plate and walls of carburetor venturi. 

Under normal circumstances he can *ot. 

Reduce speed below V^, maintain level attitude. 



48. You should select an alternate static source. 

49. Fuel starvation or mismanagement. 

50. By a suction relief valve. 

51. Airspeed would Increase, altimeter and VSI remain unaffected. 

52. Use a magnetic compass on a heading of south. Or "home" to a distant 
station on ADF. 

53. Any altitude. 

54. 32* F or colder. 

55. Apply full carburetor heat to provide alternate scmrce of air for the carburetor. 

56. Enrich mixture, open cowl flaps, reduce power. 

57. Carburetor ice. 

58. Altimeter, VSI, and airspeed indicator. 
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six Pilot Knowledge Areas Covered II Open F orm Survey 


Area 

I. Engine and Engine Operations 
n. Fuel Systems 
m. Electrical Systems 
IV. Cockpit InstrumenUtion 
V. Weather 

IV. General and IFR Procedures 


18, 19,21,22,24,25,38,39,42,45,46,49,53,56,57 
1.2,7,17 


5,6,8,11,13, 14,15,16,32,33,34 


3,4,12,20,23,26,30,31,35,41,43,44,48,50,51,58 

9,10,27,28,54 

29,36,37,40,47,52,55 
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APPENDIX E 


Closed Form Knowledge Survey With Answer 
Key and Description of Three Pilot Knowledge Areas 



A/C SYSTEMS SURVEY 


Circle Correct Answer 

(Name ) 


1. What is the standard adiabatic lapse rate? 

a. 2®F per 1000 feet - - 

b. 2H*F per 1000 feet 

c. 3"F per 1000 feet 

d. 3h“F per 1000 feet 

e. 4®F per 1000 feet 

2. Do the indications of a normally operating alternator system 

change during the course of a flight? (Assume charge -discharge ammete; 

a. Yes: Ammeter shows more charge when electrical equipment 

turned on. 

b. Yes: Ammeter shows less charge when electrical equipment 

is turned on. 

c. After engine start, the ammeter shows a higher than normal 
rate of charge and gradually declines to normal rate. 

/i. No, does not change. 

3. If an alternator is removed from the rest of the electrical 
system by an automatic circuit protection device (i.e., an 
auto reset circuit breaker) , how can the pilot attempt to 
bring the alternator back into service? 

a. Turn off the alternator and turn it back on. 

b. Turn off all switches and push reset button. 

c. Turn off switches one by one and push the reset after 
each switch is turned off. 

d. Just push the reset button. 

4. If smoke rs noticed in the cabin, what indicators would tell 
the pilot that it is electrical in origin? 

a. Discharge on ammeter 

b. Odor 

c. Erratic instrument indications 

d. Circuit breaker popped 
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5. In what range of temperatures is one most liXely to encounter 
carburetor ice? 

a. 32“F or below 

b. 20-40"F 

c. 32-80*F 

d. 50-80*F 

6. What airspeed and altimeter errors are associated with the 
use of an interior alternate static source in unpressurized 
airplanes? 

a. Airspeed and altitude read higher than actual 

b. Airspeed and altitude read lower than actual 

c. Airspeed reads higher and altitude reads lower than actual 

d. Airspeed reads lower and altitude reads higher than actual 

7. If it becomes necessary to continue operation with the use of 
full carburetor heat, what action, if any, should be taken 

to insure smooth engine operation? 

a. Enrich mixture 
fb. Lean mixture 

c. Avoid high power settings 

d. No action required 

8. What do ground based and airborne weather radar systems detect 
and display? 

a. Precipitation 

b. Clouds 

c. Thunderstorms 

d. Turbulence 

9. Which gyro flight instrument is most likely to be the last 
to tumble? 

a. Artificial horizon 

b. Turn and bank indicator 

c. Directional gyro 

,d. Depends on maneuver 
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10. Master switches, on aircraft with alternator systems, usually 
consist of two switches labeled "battery" and "alternator," 
which can be turned on or off. Which of these two switches, 
when turned off, automatically turns off the other side as 
well? 


a. Battery 

b. Alternator 

c. Neither: both must be turned off 


11. What determines the configuration of cowl flaps during a climb? 


a. cylinder head temperature 

% 

b. Exhaust gas temperature 

c. Oil temperature 

d. Outside air temperature 

12. Which of the following is the 
ceiling of an aircraft? 


a . 

Maximum 

altitude 

at 

which 

b. 

Maximum 

altitude 

at 

which 


Maximum 

altitude 

at 

which 

d. 

Maximum 

altitude 

to 

which 


best definition of the service 

the A/C can maintain level flight 

the A/C can climb 100 fpm 

the A/C can climb 500 fpm 

the A/C can climb 


13. If there is a loss of oil pressure, how would a constant speed 
propeller be affected? (Assume a non-counterweight type propeller) 

a. It would not be affected 

b. It would move to high rpm 

c. It would move to low rpm 

d. It would vary between high and low rpm 

14. Which of the following is the best procedure if you encounter 
severe turbulence in IFR conditions? 


a . 

b. 

c. 

d. 

e. 


Reduce speed below 
Reduce speed below 
Reduce speed below 
Maintain speed and 
Maintain speed and 


V^, maintain level altitude 
V^, pull on carb heat 
V^, maintain level attitude 
level altitude 
level attitude 
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15 . 


Suppose your pitot tube became totally and rapidly blocked 
(i.e., ice) at 5000 feet while you were climbinc to 10,000 feet. 
Assuming the aircraft's true airspeed remained the same during 
the climb from 5000 feet, what indications would you expect 
from your airspeed indicator, altimeter, and vertical speed 
indicator? 

a. Airspeed decrease altimeter and VSI show clinvb 

b. Airspeed increase altimet er and VSI _show climb 

c. Airspeed decrease altimeter 5000 and VSI zero 

d. Airspeed increase altimeter 5000 and VSI zero 

16. Assuming the failure of all gyroscopic instruments, how can 
pilot directionally control an airplane in a descent through 
an overcast? 

a. Reduce power, trim for 500 fpm descent 

b. Use turn and bank indicator for directional control 

c. Call approach control for vectors 

d. Use magnetic compass on a heading of south 

17. At what altitude should leaning be attempted under normal 
cruising power (less than 75%) in a direct drive, fixed pitch, 
float type carburetored airplane? 


«a . 

3000 

feet 

MSL 

b. 

5000 

feet 

MSL 

c. 

6000 

feet 

MSL 

d. 

Any , 

altitude 


18. In what free air temperature would a pilot expect structural 
icing to occur 

a. 32*F or below 

b. 32“to +40*F 

c. -10 to 32"F 

d. Above 32*F 

19. If you' suspect that your air filter has become obstructed or 
iced -over in flight, what action should you take and why? 

a. Apply full carburetor heat to melt ice 

b. Apply full carburetor heat to provide alternate source of air 

c. Lean mixture so less air would be required 

Land immediately: can't melt ice on filter 
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d. 



20 . 


What is the best method for temporarily correcting pre-ignition 
and detonation? 

a. Lean mixture and reduce power 

b. Enrich mixture and reduce power 

c. Lean mixture and increase power 

d. Enrich mixture and increase power 
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ANSWER KEY FOR CLOSED FORM SURVEY 


1. D 

11. A 

2. C 

12. B 

3. A 

13. B 

4. B 

14. C 

5. C 

15. B 

6. A 

16. D 

7. B 

17. D 

8. A 

18. A 

9. B 

19. B 

10. A 

20. B 


A 38 


Three Pilot Knowledge Areas Covered In Closed Form Survey 



Area 

Pertinent Questions 

Code* 

I. 

Engine and fuel systems 

5,7,11, 13, 17, 19,20 

CATSCR 1 

n. 

Electrical systems and cockpit 
instrumentation 

2,3,4, 10,6,9, 15 

CATSCR 2 

m. 

Weather and IFR operations 

1,8, 18, 12, 14, 16 

CATSCR 3 


♦For reference to data in Chapter VI, also see Glossary Table VI-3. 



APPENDIX F 


GAT-1 FMS Performance Profiles 



SCENARIO: #1 
SUBJECT: #1 


ACTION: 1) Returned to 
Seaport 2) Shot ADF approach 


Subject #1 is a low-time (400 hours) GA private pilot with very little instrument 
experience (7 hours actual). He flies infrequently and strictly for pleasure. 

Subject #1 exhibited difficulty in performing basic stick and rudder skills. He 
has difficulty holding altitude and heading and devoted much of his cockpit 
attention to scanning charts. He appeared nervous as exhibited by a constant 
stream of remarks to himself .^nd occasional whistling. When power failure 
occurred he immediately reached for the fuel selector and restored pKJwer. 

During debriefing he stated that he noticed that the fuel gauge was reading low. 
Furthermore he had had a similar experience before which accounted for his 
fast diagnosis. 

Subject #1 was quick to Inform ATC of his difficulties and required considerable 
guidance to successfully complete the mission. His overriding concern seemed 
to be to return "home" just as quickly as possible. 

Subject »*rs management style might be characterized as 1) reactive (he did 
not seem to have well thou^t out alternatives); 2) dependent (he asked for 
and received considerable aid from ATC); 3) repetitive (he returned to his 
departure point because he viewed that as a "conservative" and was familiar 
with current procedures and weather at that point). 
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SCENARIO: #1 
SUBJECT: #2 


ACTION: 1) Proceeded 
direct to MountaiDdale 
per night plan 


Subject #2 Is a blgh-time GA pilot (5000 hours) with most of his flight experience 
being in single engine aircraft operating In VTR conditions. Although he once 
Hew professionally his activities In recent years have been limited to pleasure 
Hying and occasional business trips in a C172. 

Subject #2 exhibited a hig^ degree of stick and rudder competence. His only 
pre-CIFE difficulties stemmed from failure to note a change in clearance and 
garbled communications. The onset of the CIFE was initially diagnosed as an 
icing problem. \\Tien carb heat failed to restore power he immediately went to 
the fuel selector and successfully restored power. 

Subject #2 was extremely self-reliant. At no time did he Inform ATC that he 
was having difficulty. He had faith In his diagnosis of fuel siphon on one tank 
only and in his calculated fuel consumption (evidenced in dtjrlefing). He con- 
tinued to his planned destination without consulting the area chart for potential 
alternates . 

Subject #2's management style might be characterized as one which Involves 
1) thorough pre-night preparation: 2) rigid adherence to plans (he carried 
through the entire mission as planned); 3) self-reliance; and 4) minimum external 
source Information seeking. 



SCENARIO: #1 
SUBJECT; #3 


ACTION: 1) Divert to 
Link Co 2) Shot VOR 
approach at Link 


Subject 1*3 Is a medium time pilot (1200 hrs. ) with most of his experience 
In military helicopters and flxed-wlng single engine aircraft. His current 
GA flying is related to occasional business trips in a C-182. 

Subject *3 exhibited extreme difficulty In basic aircraft control. Aircraft 
heading and altitude excursions had both hi^ frequency and high amplitude. 

He appeared to be overloaded and expressed that feeling during debriefing. 

His first reaction at the time of engine failure was to declare an emergency 
and dial in the emergency transponder coda. He lost nearly 1600 feet altitude 
before he was able to restore power to the aircraft. 

Subject **3 did not seem to be well prepared with basic operating Information 
and skills. During his preflight planning he commented fbat he was not familiar 
with ILS procedures. He elected to shoot a VOR approach at uncontrolled 
airport because it appeared simpler than an ILS. Furthermore he noted that the 
ILS equipped airport was In an area on the chart with many airways and VOR 
stations which he \iewed as complications, not aids. 

Subject #3's management style could be characterized as 1) reactive (he did 
not have preplanned alternatives and was unsure of his position): 2) dependent 
(his first act after the CIFE was to declare an emergency and ask for help); 
and 3) perceptually basic (he elected an alternative which looked simplest 
on the charts). 


A 43 



SCENARIO: #1 
SUBJECT: #4 


ACTION: 1) Started for 
Mountalndale, 2) Returned to 
Link, 3) Shot one missed 
approach 


Subject #4 is a professional GA and military pilot with nearly 9000 hours PIC 
time. He is currently active, flying a King Air and military helicopters. He 
is also rated as a mechanic (AI) and has built Ms own hrmebuilt aircraft. 

Subject #4 exhibited a high degree of stick and rudder competence In addition 
to being constantly aware of the state of the system. He frequently cross- 
checked his position and asked for enroute weather updates. He was well 
prepared concerning operating information. As noted in the debriefing session 
he knew precisely what the MEA*s were at the point of the CIFE, what ’^ac.lal 
he was crossing and what his theoretical fuel consumption should ha"? beer. 

He was monitoring the fuel gauges and suspected a fuel leak prior t*" actual 
engine failure. 

Subject *4 was self-reliant. He never declared an emergency and only 
Informed ATC of his difficulties after he had diagnosed the problem and selected 
an execution strateg>’. 

Subject #4's management style might be characterized as one which Involves 
1) thorough preparation, (memorized critical numbers from charts and 
manual): 2) constant information seeking and comparing; 3) flexibility (changed 
options in light of new Information concerning weather and fuel consumption); 
and 4) personal control limits (executed missed approach because fligdit parameters 
were out of his personal tolerances). 
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SCENARIO: #2 
SUBJECT: #5 


Subject #5 is a commercial pilot with multi- and single engine land and 
certlfed flight instructor ratings with fifty hours in the last ninety days of 
which four were instrument hours. lie has a total of 1550 hours. He is in his 
late 20 's and is employed as a flight lnstru''tor. 

The pilot departed from Seaport Beach airport. He experienced a decrease 
in power at Thermal Intersection and proceeded to land at Singer Airport. 

The flight proceeded routinely from Seaport to Thermal intersection. The 
pilot's ground track was good. He exhibited goc:^ stick and rudder performance. 
Difficulties were experienced with communication. At the first report of tur- 
bulence, the pilot decreased power to slow the aircraft down. North of Thermal 
intersection the pilot experienced a lubtle rpm loss. This was detected by 
the pilot visually. He was also sensitive to the fact that the temperature gauges 
were changing; both the cylinder head temperature and oil temperature. Upon 
debriefing the pilot Indicated that he was very slow to incorporate diagnostic 
changes to isolate or fix the problem, though eventually he did switch tanks, 
switch the mags and adjust mixture and carburetor heat. In fact, he left the 
carburetor heat on throughout the flight. The aircraft began to have altitude 
problems immediately. The pilot declared an emergency. He was convinced 
the airplane would not fly much longer and proceeded to ask for an off-alrport 
landing. Airspeed was maintained at about 75-80 mph. The pilot pulled the 
throttle back because he didn't want to full bore a sick engine. The rpm decrease 
was stabilized at 1500 rpm. As the pilot began to lose altitude, the researchers 
decided to open Singer airport for a landing and after much persuasion by the air 
traffic controller convinced the pilot to attempt to fly to Singer where he 
successfully landed the aircraft. 

During the debriefing the pilot felt that he was not in complete control despite 
his familiarity with this kind of equipment. He exhibited deviations in headings, 
and, as indicated above, made no attempt to keep the aircraft at the maximum 
altitude. His diagnosis was that the aircraft probably had induction icing, which 
is the reason that he kept the carburetor heat on. His management style might 
be characterized as becoming convinced that the airplane could not fly much 
longer and setting himself for an emergency landing regardless of the potential 
to use altitude trade-offs to get to nearby airports. 


SCENARIO: #2 
SUBJECT: #6 


Subject #6 is a 5,000 hour commercial aircraft, single, multi-engine, land- 
rated pilot. He has had experience in turbine aircraft. He has had two hundred 
hours total instrument, ten hours in the last thirty days and fifty hours in the last 
ninety days. His most recent IFR training flight was June, 1979. He typically 
flys a Cessna 182RG. He has over 1500 multi-engine, single pilot experience. 

He also is an air traffic controller. 

Subject #6 took off IFR from Seaport to Mountalndale. At Thermal Intersection 
he encountered power difficulties. He proceeded on and landed at Mountalndale 
airport. 

The pre-CIFE part of the flight was normal with the pilot exhibiting good 
stick and rudder control. He was very conscious of weather and frequently asked 
for weather updates at Mountalndale throughout his flight. As he began to 
encounter moderate turbulence south of Thermal, he requested and received clear- 
ance to 9,000 feet. As he climbed through 8500 the CIFE was initiated with a 
gradual reduction in rpm. The pilot noted that the rpm did not return to the 
initial level. Thus he began to assess the fact that the aircraft was losing 
power. He Initiated immediate action in terms of changing fuel selector fiwltch, 
switching the mags and using carburetor heat. He was conscious of uic Increased 
temperature for cylinder head and oil but was relieved that there was no change 
in oil pressure. This led him to believe that the problem was not basically an 
engine problem, or at least an oil loss problem. The pilot's flying strategy 
changed at the time of the CIFE to very slow flight, 60 mph, in order to main- 
tain his altitude as much as possible. He had made his own decision to proceed 
direct to Mountalndale, but alerted the air traffic controller of his problems. 
(Incldently, he felt the air traffic controller was not particularly sympathetic 
or helpful to his problem. For example, he was concerned with the minimum 
vectoring altitude around Mountalndale, which he claimed he never received. ) 

He was conscious of wind conditions being in his favor throughout most of the 
flight and decided that he had a good twenty- to thirty minutes with the aircraft 
operating at 1500 to 1700 rpm. 

The pilot stated upon debriefing that he knew he could maintain 2,000 or 
3,000 feet with 1700 rpm. His main concern wr.s getting over the mountains. 

One interesting aspect of the flight occurred when the pilot reached the 
Mountalndale area. He elected to pull the power to lose altitude rather than 
to trade altitude for air speed. His decision was explained on the basis that 
he wanted to make his approach as slow as possible because he did not want 
to execute a missed approach. This pilot's management style was 1) sticking 
to his original flight plan, 2) eliminating turning back into the wind by returning 
to Link or to Seaport, and 3) determining it was unacceptable to seek a landing 
elsewhere since the area was IFR. 


SCENARIO: #2 
SUBJECT; #7 


This subject was a 3,000 hour commercial CFI instrumented rated pilot 
who had had considerable experience in Navy Jets, multi- and single engines. 

He had had limited experience in light alrr raft and only four hours of total 
flying in the last ninety days , of which there were no hours of Instrument 
flying. His last Instrument proficiency check was two years ago. 

His initial preparation time before the flight was approximately forty 
minutes, which subsequently he argued was insufficient for his current exper- 
ience in IFR flying. Examination of the pilot controller tapes and initial pre- 
CIFE navigation imd flight control indicates that the pilot did a good Job in 
complying with his clearance, in communic:Uion and navigation and was rated 
high in terms of aircraft attitude control. 

Twenty-five minutes into the flight, after reaching Thermal Intersection, 
the pilot experienced a gradual decrease in rpm from 2500 to 1500. At that 
time he was cruising at 6,000 feet. He asked for an Immediate descent to 
the nearest airport but was reminded by Center that the area was currently 
IFR. As he proceeded to lose altitude, holding his airspeed at approximately 
85 mph, he reported at 5,000 feet, at which time the controller asked him if 
he wished to declare an emergency, upon which he agreed. Analysis of his 
ground track showed erratic heading maneuvers, including a 3G0'' maneuver 
northwest of Thermal. 

The pilot, after asking for weather, and rejecting the idea of continuing 
the flight over the mountains to Mountalndale decided to divert to Link County 
airport. At that time he was losing altitude and was at 4500 feet. Because 
of the concern by the research staff of his ability to get to Link, with his current 
air speed and rate of descent, it was decided to have another light aircraft report 
out of Singer airport that it was turning VFR. At the Center's suggestion the 
pilot opted to go direct to Singer. His riltltude at that time was approximately 
3,500 feet. 

The pilot, four minutes after the CIFE, reported his concern that the 
engine instruments were reading high :uid said that he could "anticipate" engine 
failure. Upon the direct clearance to Singer, the pilot requested both vector 
and headings distance to Singer, as well as field elevation In the event of 
possible radio failure. The pilot proceeded direct to Singer at 3,500 feet, and 
successfully huided the aircraft. 

The debriefing of this pilot is particularly insightful and Indicates that the 
pilot had an acute sense of the failure of his own performance in this situation. 
His major concern was his lack of preparation for procedures to handle what 
he conceived to be an icing problem. He detected the rpm loss visually, not 



aurally and confirmed It upon application of carburetor heat. In which case the 
rpm did not return to Its original level. He was particularly conscious of air 
speed and altitude; so much so that he "admitted” that he lost his scan capa- 
bility. As a result of this his heading became erratic Immediately after the 
CIFE. The pilot admitted, upon debriefing, that he literally turned the 
navigation of the airplane over to Center. 

His Immediate concern following the CIFE was to get the aircraft on the 
ground as soon as possible. After the aircraft maintained 1500 rpm for several 
minutes he believed he could maintain 1500 rpm and 3, 500 feet In proceeding 
to his new destination. He was not particularly conscious of his air speed. 

The relatively high air speed the he selected (85) caused a descent rate that 
prompted the researchers to open up an emergency landing field for the pilot. 

Since his problem occurred right after switching the tanks, his first concern 
was that there was a fuel problem. However, the fact that the rpm could be 
maintained at 1,500 dissuaded him from this particular position. The pilot, 
on debriefing, was not sure why he did not slow up the airplane for altitude. 

He was ashamed that he was not aw’are of the position of Singer airport on his 
chart. The pilot was sensitive to Increasing temperatures for cylinder head 
and oil temperature, and this created a sense of urgency on his part for getting 
the airplane down. He tried to get out the flight manual for the aircraft to 
determine proper procedures for icing. He was resigned to bringing the air- 
craft to an off airport landing. 

This pilot diagnosed bis problem to be one of inadequate preflight prepara- 
tion In terms of alternate airports and preparation for handling procedures In 
the event of icing (which he had diagnosed as his problem). Thus, with the 
introduction of the CIFE he became overly absorbed in his altitude control, 
lost his basic scan pattern and proceeded to develop heading problems. With 
all of this workload upon him the pilot essentially opted to turn the navigation 
decision over to Center. Indeed, It was the controller who recommended Singer 
airport as an alternate for the pilot. The pilot had agreed with the debrlefer 
that the situation was very realistic and, in fact, was worried that he might 
develop vertigo with his heading diversions Immediately following the CIFE. 
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SCENARIO; #2 
SUBJECT: #8 


Subject #8 is a newly-minted IFR pilot with approximately 250 hours. He is 
thirty years old and took up flying aft^r graduation from college. Most of 
his experience has been in fixed wing, fixed pitch, fixed gear aircraft. He 
currently flies a 172. He has had considerable experience in the simulator. 

His preflight might be considered extremely ponderous and slow as the pilot 
sat in the cockpit for almost fifteen minutes before initiating takeoff. He 
seemed very meticulous in terms of organization of cockpit housekeeping. 

The pilot was IFR from Seaport to Mountaindale. Upon the Initiation of 
the CIFE eventually he was radar- vectored to Singer airport for a landing. 

His pre-CIFE fli^t might be characterized as average in terms of ^tick 
and rudder control. The pilot apparently had only minimum consciousness 
of his position in the air space. This was illustrated by his inability to know 
the Link VOR frequency, one of the major VORs along his path. The aircraft 
was cruising at 8,000 feet near Thermal Intersection when the CIFE was initiated. 
The CIFE was a gradual decrease in rpm. The pilot detected this aurally 
and initiated carburetor heat action. It was clear that the pilot became rapidly 
overloaded and lost scan patterns because there was considerable drifting in 
heading. His first request was for a precautionary landing at the nearest air- 
port. When advised that the situation was IFR, the pilot attempted to proceed 
on to Mountaindale. He was reminded by air traffic control of the MEA in 
the region. As his altitude began to fall, the pilot exhibited extreme measures 
in air speed to keep altitude, frequently tripping the stall alarm, and at approxi- 
mately 7,000 feet the pilot, at the point of declaring an emergency, clearly 
turned the navigation part of the flight over to the air traffic controller, with 
a statement, "Get me down". At this point it was decided to open up Singer 
airport VFR so that vectors could be given to him to proceed directly there. 

The debriefing of this pilot suggests that he was unprepared for the emergency 
and admittedly was overloaded. His concern with altitude and air speed and rpm 
resulted in him spending a considerable amount of time giving information to the 
air traffic controller, seemingly looking for the air traffic controller to provide 
not only recommendations on where to go but what to do. it was the air traffic 
controller who made the decision not to proceed up into the mountains with 
decreasing altitude. It was the air traffic controller who was conscious of the 
MEA. It was the air traffic controller who ultimately had to suggest to the pilot 
that he ml^t proceed to Singer, even though this information was now avail- 
able to the pilot. His management style of the CIFE might be characterized as 
one of total preoccupation with one or two aircraft parameters and the inability 
to "put together" a reason/ decision about the nature of his problem and what to 
do about it. He admitted on debriefing that he tried to do what he could to get 
some more rpm and when he concluded that there was nothing that he could do 
about it he was less concerned about the possible diagnosis of the failure. 
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SCENARIO: #3 
SUBJECT; #9 


ACTION: 1) used ATC to 
confirm failure, 2) attempted 
one VOR approach; missed, 

3) 2nd VOR approach was 
successful 


Subject #9 Is a flight engineer for a major airline. The majority of his 
flight experience was gained in the military where he piloted single-engine 
turbojet aircraft. He is still active in military aviation as an Air Guard 
pilot and instructor. In GA aircraft he has earned FA A certification as a 
flight instructor. 

Subject #9*s basic stick and rudder skills were highly refined. His 
movements of controls were well planned and smoothly executed. Excursions 
from assigned headings and altitudes were minimal, even during the CIFE. 
Beyond the basic piloting of the aircraft, he had plenty of spare capacity to 
perform navigation and communication duties. During the first ILS approach 
he understood and complied with all ATC Instructions, and conducted the 
approach down to the decision height before executing the missed approach. 
During the first approach, deviations from the glide slope and localizer center- 
lines were small, and the entire approach was conducted in a stabilized con- 
dition . 

The CIFE started as the flight was being radar vectored for a second 
approach. The flight was given a final heading to intercept the localizer course 
when the localizer needle was covertly rendered inoperative. (In the failed 
mode the localizer needle moves to the center and remains motionless and 
there are no other cues that Indicate the localizer has failed. ) When he first 
saw the needle in the center, he thought he may have already Intercepted the 
localizer course and was passing through it. Just a few seconds later, how- 
ever, he asked ATC for his position relative to the localizer course, and, 
noting the discrepancy, became suspicious of the Instrument. Using radar 
vectors and position updates, the subject confirmed that the needle had failed 
within 1 l/2 minutes after the onset of the problem, and reported his situation 
to ATC. 

Subject #9's next action was to choose an alternate type of approach. His 
first choice was an ASR approach, but that was not available at Mountalndale. 

He next considered an NDB approach, but because of intermittent and Inaccurate 
ADF indications he decided to proceed to the VOR and shoot the VOR approach. 
Before making this decision, he checked the local winds at Mountalndale to see 
if this approach would be feasible. 

Subject #9 conducted one VOR approach but executed a missed approach 
early in the procedure when he became suspicious of the VOR needle indications. 
His suspicions were unfounded, however, and he successfully completed a 
second approach and landed. 
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Subject #9's management style might be characterized as "aviate, 
navigate, communicate", ’^ecise control of the aircraft was maintained 
at all times. He was always aware of his geographical position and used 
all navigation aids "in concert" to follow his progress. He also made use of 
ATC services in an efficient and organized manner. His relationship with 
the radar controller was one of a cooperative effort, but he always reserved 
final responsibility for the operation of the aircraft. The authoritative manner 
in which he requested ATC assistance was indicative of his understanding 
of the ATC system and it’s capabilities and limitations. It was also evidenced 
that he used ATC resources as an aid, rather than as a means to "bail him 
out". 
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SCENARIO: #3 
SUBJECT: #10 


ACTION: 1) did not use ATC to 
aid in diagnosis, 2) tried NDB 
approach but went far off-course, 

3) wanted another NDB but per- 
suaded to do otherwise by ATC, 

4) successfully completed VOR 
approach. 


Subject #10 is a private pilot with an instrument rating who has very 
little experience. He has a total of almost 300 flight hours, all of which 
has been in light single-engine aircraft. At the time of this experiment, 
he had only recently acquired his instrument rating and did not have any 
instrument experience beyond that required for the rating. 

Subject #10's basic stick and rudder skills were considered to be fair, 
with occasional deviations from assigned headings and altitudes observed 
during unstressed periods. At times when cockpit workload and distraction 
were higher, larger deviations were observed. His communication skills 
were also fair. He understood most transmissions and acknowledged using 
standard phraseologj'. His navigation skills, however, were somewhat less 
than polished. On the first ILS approach, he had much difficulty bracketing 
the localizer course, and misinterpreted glide-slope indications. During the 
post-flight briefing, he mentioned that he thought an ’Hip" glide-slope needle 
meant a "down" correction was needed to get back on the glide-slope. 

(This is the reverse of what is true. ) Following this notion he continued 
to descend to intercept the glide-slope. As a result, he was always below 
the glide-slope during the first approach. After the glide-slope needle had 
not wavered from its full "up" Indication for awhile, he thought it mi^t be 
faulty and asked for clearance to conduct a localizer approach. When the 
flight was about one mile outside the outer marker, it was so cleared. The 
flight reached the outer marker near the localizer course and 300 feet below 
the minimum altitude published in the chart. From that point, the subject 
was unable to track the localizer and wandered well to the right of course 
while reaching the MDA. Finally, when still about tw'o miles from the air- 
port, he confessed to the tower controller that he was "totally off course and 
lost track of time" and that a missed approach would be necessary. 

During the second approach, the CIFE was introduced as the localizer was 
secretly failed as the flight was given a heading to Intercept the localizer course. 
Shortly after the localizer was failed, the subject sensed something was wrong. 
He noticed the localizer needle was centered and motionless. (For a pilot who 
was totally unable to center the needle during the previous approach, this indica- 
tion was, understandably, a cause for suspicion.) He switched frequencies on 
his localizer receiver, and, noting no change in needle position, assumed the 
system had failed. 
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Because of his position relative to other navigation aids, the subject chose 
to proceed to the beacon and conduct an NDB approach. The approach controller 
cleared the flight for an NDB approach and advised the pilot that field conditions 
were "500 feet overcast and two miles visibility with rain and fog". Though 
the stated visibility was above the legal minlmums for landing, the low reported 
celling made visual contact with the ground during the approach unlikely. 
Nevertheless, the flight proceeded to the beacon, crossed it 350 feet below the 
published minimum altitude, and commenced a descent to the MDA. Shortly 
thereafter the flight had wandered well to the right of course and was 260 feet 
below the MDA. The pilot was not aware of how seriously he was straying 
from the approach procedure and executed a missed approach only after be 
was instructed to do so by ATC. 

Still not aware that the weather was not suitable for such an approach, 
the pilot requested another NDB approach. As the flight was preparing for a 
second NDB approach, the radar controller advised the flight that current 
weather conditions (500-2) were not favorable for the successful completion 
of the approach. At this point the pilot decided to proceed to the VOR and 
execute a VOR approach. The flight was given vectors for the final approach 
course, and completed the approach and landing in a routine manner. 

The subject displayed no clear management style or priority during either 
the routine phase of the fli^t or the CIFE. Although basic aircraft attitude 
control was adequate (l.e. the aircraft was never in serious danger of stalling 
or assuming a critical attitude), on two occasions he did allow the aircraft 
to wander into dangerous situations by not adhering to course and altitude 
restrictions. His ability to conduct ILS approaches was severely hampered, 
of course, by the erroneous impression he had about the way glide-slope 
information is presented. Also, the last and eventually successful approach 
(the VOR approach) was the only one where he d’ , not stray far from the 
approach course. It was interesting to note that the subject did not request 
any special ATC assistance to determine the nature of his problem or how to 
resolve it. Though he appeared to be familiar with most of the phraseology, 
communications were limited to the customary exchanges between pilots and 
controllers. In radio conversation and from the direct observation of pilot 
activity during the simulation, he did not exhibit any outward sign of nervousness, 
anxiety or being overloaded. 

In summary, it seems that although the subject was able to communicate 
and control the aircraft at, at least, a minimally acceptable level, he was unable 
to make crucial navigation decisions in terms of headii^ and altitude selection. 
This inability' could be attributed to inexperience, poor training, or the pilot 
merely having a bad day. 
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SCENARIO: #3 ACTION: 1) ATC was largely 

SUBJECT: #11 responsible for pilot's detection 

of problem, 2) pilot chose the VOR 
approach to complete mission, 

3) while enroute to VOR, pilot 
thought localizer needle was operating 
properly again and wanted to shoot 
another ILS approach, 4) due to 
anticipated delays, pilot changed mind 
again to the VOR approach, executed it, 
and landed. 


Subject #11 is a relatively new pilot who holds a commercial pilot certificate 
with instrument and multi-engine ratings. He is also a certified flight 
instructor and works part-time as such for OSU’s Department of Aviation. 

He has acquired about 500 hours of total flight time, but his Instrument flight 
experience does not extend far beyond that required for the Instrument rating. 

The subject was fairly proficient at the basic control, navigation, and 
communication skills. Control of the aircraft was maintained in a smooth and 
coordinated manner. Deviations from assigned headings and altitudes were slight. 
Though some difficulty with radio communication equipment was encountered, 
the subject usually did understand, reply, and respond projjerly to radio trans- 
missions. His ability to navigate was also ?t an acceptable level. On his first 
ILS approach, he had captured the localizer course and glide-slope after the 
usual bracketing maneuvers. From that point, the first approach was conducted 
to within 100 feet of the decision height, with a few deviations occurring around 
the glide-slope and localizer course. 

On the second approach, the CIFE was introduced just as the flight was getting 
established on the localizer. The failure of the localizer needle was so subtle 
that the pilot never noticed it, continued to the outer marker, and Intercepted 
the glide-slope. Still thinking all was well and that he was on the localizer 
course (from the indications of the instrument in the failed mode), he continued 
to descend on the glide-slope to the vicinity of the middle marker. On at least 
two occasions during this descent, the flight turned to the right to headings which 
should have resulted in off-course indications from the localizer needle. In 
addition, the subject received three ATC warnings, since crossing the outer 
marker, that the flight was observed, on radar, to be well off course. The 
subject, however, continued to descend and executed a missed approach only after 
he w'as instructed to do so by ATC. When he started l.is missed approach he was 
less than 200 feet from the decision height, crossing the middle marker, and 
was well to the right of course. 

As he was climbing on his missed approach, the subject informed the radar 
controller that his localizer must have failed and that he was requesting the VOR 
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approach. A few minutes later, however, he thought the localizer might have 
resumed normal operation and wanted to make the ILS approach. After being 
informed of traffic delays for the ILS, he decided again on the VOR approach 
and landed without further Incident. 

Though he seemed proficient in the basic skills, the way in which he man- 
aged this problem raises a few questions. In the post-flight briefing, he 
said he first felt something was wrong when ATC first advised him he was 
off course while his own localizer needle Indicated he was on course. He 
also said he made a few sli^t turns to see if the localizer needle moved. 

It is not clear, then, why the subject continued to descend another 600 feet 
In the one minutes that followed the first ATC warning, or why he continued 
to descend after the second and third warnings. Presumably, the additional 
warnings should have lent more support to the hypothesis that some serious 
navigation problems existed. Also unclear is the reason why he thought the 
localizer may have started working again after he Initiated the missed approach, 
and why he wanted to try another ILS. (Recall that he was discouraged from 
trying the ILS approach due to traffic delays. ) 

The subject’s apparent style of fli^t system management was to assume 
everything was operating until there was substantial proof otherwise. Until 
he had the evidence to prove beyond doubt that the localizer had failed, he 
continued to trust it and to descend on an approach into a mountainous area. 
Also, he first had to be convinced that the localizer was inoperative before he 
considered other alternatives. Even after three ATC warnings and being 
Instructed to execute a missed approach, he was not overwhelmingly convinced 
that the localizer had failed because he considered shooting the ILS again. 

From this, one must conclude that either the subject did not lend much credence 
to the ATC warnings, or he really wanted to believe his localizer was still 
working. 

This school of thou^t is not an entirely healthy one, and it contradicts 
the more conservative philosophies prevalent in flight training circles. In 
flight training it is desirable for the student to gain a basic level of suspicion 
about the airworthiness of an aircraft and its systems. This is not intended 
to undermine the trust a pilot has in his aircraft. It does serve, however, as 
an incentive for the pilot to require proof of the integrity of aircraft instruments 
and systems through preflight checking, crosschecking indicators, and 
monitoring trends . 
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SCENARIO: #3 ACTION: 1) became suspicious of 

SUBJECT: #12 localizer needle but continued approach, 

2) assumed localizer failure when ATC 
told flight it was off-course, 3) chose 
to conduct VOR approach and completed 
It (with difficulty) 


Subject 12 Is a pilot of average experience. He holds a commercial pilot 
certificate with an Instrument rating. He is also a certified flight Instructor 
and is employed part time as such for OSU's Department of Aviation. All of 
his flight experience has been In ll0it single-engine aircraft, and his Instru- 
ment experience extends slightly beyond that required for the rating. 

Subject #12 was not very proficient In the basic communication, navigation, 
and aircraft control skills. On a few occasions he missed entire radio messages 
which were Intended for him even though there were no apparent problems with 
his airborne equipment. At other times he did understand and comply with die 
bulk of the Instructions but it was necessary to repeat some of the details for 
him. In controlling the aircraft he allowed some fair-sized oscillations to 
occur around assigned headings and altitudes. He did maintain general control 
but It was not precise. The subject also had difficulty In navigating. On his 
first ILS approach he had trouble bracketing and staying on the localizer course. 
This problem was compounded by his feeling that he had an oil pressure problem. 
(In fact, no problem existed. Subject #12 may have been looking for problems 
to occur, given the nature of the study.) A short time after crossing the outer 
marker, the flight had deviated well to the right of course, when he announced 
his missed approach and his intentions to try another ILS. 

The CIFE occurred on the second approach In the form of a localizer failure 
as the flight was being vectored to the final approach course. About 1 1/2 
minutes after the failure occurred, the subject became suspicious of his Instru- 
ment and asked the radar controller If he showed the fll^t established on the 
localizer course. (During the post-flight briefing he said the apparent 
insensitivity of tJiO needle to changes in position led to this suspicion. ) The 
flight was advised that it was near the centerline, but, on Its present heading, 
would not Intercept the localizer centerline for another two miles. The flight 
then assumed a heading which paralelled the localizer but kept it slightly to the 
south of course. About two minutes later the flight was advised It was one mile 
from the outer marker, slightly to the south of course, and it was instructed 
to change frequencies. As the flight crossed the outer marker it turned to 
headings (40° to 50° to the right of course) which should have resulted in off- 
courcj indications from the localizer needle. The fli^t began to descend on 
the glide-slope but continued digressing from the localizer course. At the first 
suggestion from ATC that the flight was significantly off-'ourse, Subject »12 
Initiated a missed approach and advised the controller that his localizer must 
have failed. The subject chose to execute the VOR approach to complete the 



flight but had much difficulty doing so. A great deal of ATC assistance was 
required to get the flight Oi. the VOR final approach course. Ctoce established, 
however, the flight completed the approach and landed without further Incident. 

It was not clear what Subject #12’s method of managing the CIFE was from 
observing his actions. During the post-fiit,ht briefing he did mention that, 
once the localizer failure w'as suspected, he decided to continue the approach 
until he was certain of the failure. Once the problem was confirmed, he said 
his next action was to climb to a safe altitude and then decide what to do next. 

It was apparent at several points in the flight that he was being taxed by 
different situations to near the limits of his piloting resources. As he crossed 
the outer marker on both IL5 approaches, for example, (a point where workload 
is usually higher), heading and aircraft control became progressively more 
erratic. This overloading, no doubt, had an effect on his perception of the 
problem and the way he managed it. This may explain why the flight continued 
for several minutes in attempting the ILS approach after a localizer failure 
had been suspected. It may also have been the reason why the subject did 
not continue with any diagnostic activity once he suspected the failure. (The 
40° to 50° heading deviations with no needle change should have been enough 
evidence. ) It was not until the advisory from ATC that the flight was off- 
course that his suspicions were confirmed. 
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Figure Gl: Scenario #1, Data Summary Sheet 
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Figure G2: Scenario #1, CIFE Response 
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Figure G3. Scenario #1, Decision Factors Rating 
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Figure G4: Scenario #2, Data Summary Sheet 
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Figure G5: Scenario #2, CIFE Response 


2 

Detection mode 

Tach. Oil Temp. 

A/C performance 

Tach 


Problem Detection - Diagnosis 

Detection time 

1.3 min 

1.5 min 

1. 0 min 


Diagnostic Procedure 

carb heat 
mlxtiire 
eng. gauges 
magnetos 
throttle 

mixture 
carb heat 
magnetos 
switched tanks 

throttle 
carb heat 


Prepared strategy 

none 

none 

none 


Declared emergency? 

yes 

yes 

yes 


Info sources used 

ATC 




Perceived cause 

Induction Ice 

Internal malf. 

Icing 


Observed stress 

5 

5 

5 


Pilot's crit. est. 

7 

8 

6 


Relevant experience 

carb. Icing 

power loss 

pitot Icing 


Est. flying tine 

1. 6 hrs. 

45 min 





2 

Alternatives 

considered 

find field 
to land 

Link Co. 
Singer 

Mountalndale 

Seaport 

Link 


1 

Decision 

put down 
off-air port 

continue to 
Mounta indale 

divert to 
Link Co. 

1 

1 

n 

c 

Reasons 

wanted power 
for landing 

poor weather 

In East, no 
appr. at Singer 

Mbuntaindale 

elevation too 
hlsii 


u - 

•H 

U 

< 

Changes in plan 

land at 
Singer 


land at 
Singer 


•O ' 

c 

03 

n 

c 

1 

Reasons 

ATC suggested 
within range 


ATC suggested 
within range 


0 

•H 

n 

•H 

0 

0) 

Flying technique 

reduce pwr. , 
head Into wind 
for landing 

slow night 
tried flaps 

over controlled 
flew for airspeed, 
then altitude 


Q 

Outcome 

successful 
VFR landing 
at Singer 

successful 
ILS landing 
at Mountalndale 

successful VFR 
landing at 
Singer 

^ • 


























































iiuiuciicc Oli uccidtuli 


Figure G6: Scenario #2, Decision Factoi i Rating 
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Figure G7; Scenario #3, Data Summary Sheet 
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Figure G8: Scenario #3, CIFE Response 


Detection mode 


GDI frozen 


GDI position 


ATC advice 


GDI position 


Detection time 


0. 5 min 


1.5 min 


5. 0 min 


2 min 


n 

l-H 
0) 
10 
c . 

O' 

fl 

Q 


Diagnostic Procedure 


ck. aud. ID, 
ck. OFF flag, 
observed GDI 
on final appr. 


freq. selector 
fuses 


LOG failure on 
course, pilot 
flew LOG well 
until MM when 
ATG advised 
off course 


I 


Prepared strategy 


none 


none 


none 


c 

0 

•H 

l-P 

0 

0) 

l-p 

(!) 

IQ 


Declared emergency? 


no 


no 


no 


Info sources used 


none 


none 


ATG 


Perceived cause 


faulty needle 


radio malf'cn 


radio malf'cn 


Observed stress 


1 


Pilot's crit. est. 


IS 

1^ 


Relevant experience 


none 


none 


none 


Est. flying time 


2 hrs. 


2 hrs. 


2 hrs. 


continue until 
failure verlfit 


no 


ATC fix 


rad’.o malf cn 


ILS failure 


2 hrs. 


Alternatives 

considered 


NDB appr. 
VOR appr. 


continue but 
execute NDB, 
missed appr. 


missed appr. , 
NDB, VOR, 
Seaport 


another airp 
VOR appr. 


1 Decisions and Actions 



examine others 



Decision 

missed appr. 
execute VOR 

attempt NDB 

missed appr. 
attempt NDB 

missed appr 
execute VOF 

Reasons 

trouble with 
ADF 

not yet to 
outter marker 

? 

only feasible 
alternative 

Changes in plan 


missed appr. 
attempt VOR 

attempt VOR 
appr. 




Reasons 



15 min. delay 
for NDB 





Flying technique 

no change 

no change 

1 

no change 

no change 

Outcome 

successful 
VOR landing 
at Mountaindale 

successful 
VOR landing 
at Mountaindale 

successful 
VOR landing 
at Mountaindale 

successful 
VOR landini 
at Mountain 
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Influence on decision | i| Decision ij |! Situation' 


Figure GJ: Scenario #3, Decision Factors Rating 


Scenario #3 

Estimated flying time 
Estimated fuel on board 
Enroute weather 
Destination weather 



12 

Average 

Range 

7 

7.0 

1 

3 

4.7 

5 

3 

3.2 

2 


Scenario #3 

9 

10 

11 

12 

Average 

Range 


Conservat ism 


B 

6 

5 


4 


Safely 


■■ 


6 


4 


Match with piloting skills 


4 


5 


6 


Familiarity with A/C procedures 


2 


4 

4.0 

5 


Familiarity with NAV procedures 


5 


7 

6.2 

3 


Proximity to original intentions 


6 


6 

4. 2 

6 


Compliance with FARs 

6 

2 


7 

4.0 

7 


Flying time needed to execute 

1 

1 

3 

1 

1.5 

3 


Scenario #3 

A/C condition when decision made 

Fuel on board 

Enroute weather 

Destination weather 

Time needed to execute 

Familiarity with A/C procedures involved 

Familiarity with NAV procedures involved 

Proximity to original destination 

Compliance with FARs 

Other 


9 


11 

wm 

Average 

Ranee 

IB 

1 

6 

5 

4.7 

7 

■■■ 

1 

7 

6 

3.7 

7 

- 

2 

4 

6 

4.0 

5 

6 

1 

1 

6 

3.5 

6 

3 

6 

2 


4.5 

6 

1 

5 

5 


4.5 

7 

6 

2 

5 


3.7 

5 

ImM 

5 

- 


6.0 

3 

B 

4 

1 


3. 7 

6 

IB 

5 

6 


6.0 

3 
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Figure G15: GAT Subject Performance 
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Figure G18: GAT Subject Performance 








































Figure G21: GAT Subject Performance 














APPENDIX H 


Paper and Pencil Testing Materials 




PILOT BACKGROUND DATA 


Subject #: 

Date: 

Time: 


Pilot licenses held: 

Ratings and limitations: 

Primary flight training: 1 1 

military 

\ 1 civilian 


Type of flying done most often: 

1 ] airline ! IgA comm 

1 1 business 1 1 military 

1 1 pleasure 

Total flying time: 

hrs. 



Total small, single engine: 

nrs. 



Total JFR; 

hrs. 
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GENERAL INTRODUCTION 


Today we would like you to participate in a few paper and 
pencil scenarios. We will be considering a series of hypothetical 
flights which will take place in the Northeastern United States. 

The booklet in front of you contains some information which we will 
proceed through in a step-wise fashion. 

Page 1 shows a map of the Northeastern and Northcentral 
United States, and Southeastern and Southcentral Canada, All 
of the hypothetical flights we will consider will take place in 
the area surrounded by the dashed lines. You can see this includes 
Vermont, New Hampshire, and parts of Maine, Massachusetts, New 
York, and Quebec. 

Page 2 is a simplified weather chart of the same region. 
Pictured on this chart are the major weather systems prevalent 
during the time of these flights. As you can see, a low pressure 
system is centered over Michigan and Lake Huron, and it is moving 
very slowly eastward. A warm front extends from the low center 
northeastward, and a cold front extends from the low center south- 
ward. Our area of concern is the region within the dashed lines. 

The weather in this area has been caused by the warm front and is 
generally rainy and drizzly, with light southeast winds end reduced 
visibilities. There is no severe weather in this area. Although 
the cold front is approaching the area, it is a relatively dry 
front and should not be a factor during the time we will be flying. 

A weather depiction chart is shown on page 3. It shows the 
areas where visibilities are less than 3 miles and/or the ceilings 
less than 1000' by enclosing these areas in a solid line. Scalloped 
lines surround the areas where ceilings are between 1000* and 
5000', and visibilities are greater than 3 miles. You can see 
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that IFTl conditions prevail over most of our area of concern, 
except over northeastern New York, where conditions are slightly 
better . 


r 


These charts are intended to give you a brief overview of 
the "big" weather picture. More detailed information on the 
weather wiJl be provided when appropoiate. 

The airplane you will be flying is a Cherokee Arrow. This 
is a typical four-place plane with retractable landing gear and 
constant speed propeller. It is powered by a 200 HP, fuel injected 
engine. The airplane is equipped with standard IFR instriimentati on 
and avionics. This should be ample information for you to begin 
our paper and pencil scenarios. 
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Notes: Other Information available from Diagnosis Problem 

Interviewer: Subject # 

- visible plane exterior Experimenter _____ 

-outside air temperature 
-Interior cabin condition 





Instructions for P/P Diagnosis; 


You have been given n brief rundown on the plane, airspace and current weather. The 
next phase of our study will pro'^eed as follows: First, I will read you an introduction to a 
flight you will be taking. Then I viH read to you some symptoms which will indicate some 
form of problem with the plane you are flying. After that, I will start the clock and you 
will have 4 minutes to diagnose the problem. Referring to your diagram of the 
instrument panel, and your knowledge of aircraft systems, you can ask me for individual 
readings from the instruments and controls shown. I will respond by telling you the status 
or reading of the chosen instrument or control. You can then write this information some- 
where on your panel diagram if you wish (you'll get a clean sheet for each problem). Any 
actions you may want to take to improve the safety of the situation or to test the plane's 
reaction should also be mentioned in the sequence you would normally use. I will explain 
the effect of your actions in sequence. No multiple, simultaneous actions will be possible. 

Please think out loud as to what the possible cause of the problem might be at any 
point in your diagnosis, even if you have several potential causes in mind. 

You are working against the clock. Ask for information only if you think it will help 
your diagnosis. The information available from me concerning your panel and the cockpit 
environment is adequate for complete and unambiguous diagnosis of the given problem. Any 
information "not available" will not influence the problem. Any information termed 
"normal" means that the instrument reading is in its normal range or that the response of 
the plane to a control input is typical for that phase of the flight. 

Once you are certain that you know what the problem is tell me and I will verify your 
diagnosis. If you are wrong, you can continue until you've identified the cause or until 
time is up. If time runs out, you can make a best guess. In any case, I will explain the 
cause of the problem when the diagnosis period is over. 

We will be trying 4 different problems. Any questions ? 



FLIGHT INTRODUCTION FOR DIAGNOSIS 


Diagnosis Scenario 1 

You are making a day trip from A.bany, NY to Burlington, VT. You fly 
out of Albany at 9:00 a.m. , cl ‘f' ed Victor-91, Burlington. You climb 
to a cruising altitude of 7000 ft. after 20 minutes of routine flying you 
notice the smell of hot engine oil. What would you do? 


[READ SLOWLY - repeat any portion necessary for pilot's full understanding] 


EXPLANATION OF CAUSE FOR SCENARIO 1 


A small crack developed In the oil line feeding the oil pressure 
gauge. This crack reduced the oil pressure reading drastically, 
but did not seriously affect the actual lubrication of the engine. A 
small pool of oil began to form on the floor of the cabin, pilot's 
side. Assuming that the crarked line would not deteriorate quickly 
into a complete break, you were in no immediate danger of engine 
seizure. 
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FLIGHT INTRODUCTION FOR DIAGNOSIS 


Diagnosis Scenario 2 

You are making a day trip from Augusta, Maine to Lebanon, New 
Hampshire. You Hy out of Augusta at 9:00 a.m. , cleared Victor 39 
to Neets Intersection, Victor 496 to Lebanon. You climb to a cruising 
altitude of 6000 feet. After 15 minutes of routine flying in instrument 
conditions, your instruments indicate an increase in airspeed and 
steadily decreasing altitude while maintaining levd flight attitude. 

WTiat is the first thing you would do ? 
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EXPLANATION OF CAUSE FOR SCENARIO 2 


Your vacuum pump failed as indicated by the low reading of the 
suction gauge. The vacuum pump drives the attitude and directional 
gyros. As the artificial horizon lost its drive it started to sag to 
the right and you compensated by turning left, leveling the artificial 
horizon and putting the plane in a slow, descending left bank. The 
airspeed increase was due to the sli^c nose-down attitude. 


FLIGHT INTRODUC'nON FOR DIAGNOSIS 


Diagnosis Scenario 3 

You are making a day trip from Keene, New Hampshire to Montpelier, 
Vermont. You fly out of Keene at 10:30 a.m. , cleared Victor-151 to 
Montpelier. You climb to a cruising altitude of 5000 feot. After 20 
minutes of routine cruise your engine suddenly starts running e.ctremely 
rough, shaking the whole plane and losing about 20% of its cruise power. 
What is the first thing you would do? 
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EXPLANATION OF CAUSE FOR SCENARIO 3 


Your engine suffered a broken drive gear In ^he right magneto. 
The resultant untimed ignition conflicted with the remaining good 
ignition and caused the extremely rough engine and backfiring. 
Switching from "both" to the left magneto would have resulted in 
a smooth running engine with slightly less p)ower than normal cruise. 
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FLIGl.*’ INTRODUCTION FOR DIAGNOSIS 


Diagnosis Scenario 4 

You are making a day trip from Sanford, Maine to Messena, New York. 
You fly out of Sanford at 8:30 a.m. , cleared Victor-496 to Lebanon, 
Victor 141 to Messena. You climb to an initial cruise altitude of 6000. 
After about 20 minutes, Boston Center instructs you to climb and main- 
tain 10,000 feet. You acknowledge and begin your enroute climb 
between layers. After 2 minutes of climb, you notice your indicated 
airspeed dropping off steadily from 100 kts. , maintaining constant 
pitch attitude. What would you do? 
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EXPLANATION OF CAUSE FOR SCENARIO 4 


As you climbed through 6500 feet, the static port froze over as 
the outside air temperatiire dropped below 32* F. This caused the 
airspeed indicator to decrease as altitude increased and the VSI and 
altimeter to read low. Several corrective actions were possible: 
return to your previous altitude of 6000 feet; open the alternate static 
source; break the VSI glass. 
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DECISION PHASE 
Checklist for Each Subject 

A. Have these materials ready and in the following order: 

1. Figure I (Lo En route Chart) 

2. Tables I, II, III, and IV (Stapled) 

3. Figure II (Route Chart) 

4. Figure III (Airport Chart) 

B. Have Subject number. Experimenter name, and date filled out 
on the "Information Sheet" and be prepared to collect data 
on this sheet. 
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SCENARIO E 


You are at the Bangor International Airport in Bangor, Maine, and desire 
to By tc Glens Falls, New York, for a 1:00 p.m. ouslness ‘meeting (shown in 
Fig. I). The current time is 9:00 a.m. and you feel you can be ready for departure 
by 10:00 a.m. after you conduct all necessary preflight activities. The plane 
you will be flying today is your company's Cherokee Arrow (N8086W). You 
have flown this particular plane several times before and regard it as a reli- 
able airplane. A brief list of the important performance figures and IFR 
equipment on board is shown in Table I. (pause) The aircraft's fuel tanks are 
full, and after a very thorough preflight inspection, you conclude that it is 
operationally and legally ready for the flight. 

Now your attention turns to the weather and filing a flight plan. You cal! 
the nearest Flight Service Station on the telepihone and obtain the weather infor- 
mation in Table II. (pause) After compiling a navigation log for the flight, you 
file the flight plan in Table m. (pause) 

Based on the information you have received so far, would you normally 
attempt this flight? ("Yes" or "No" - Circle on Information Sheet, Question I.) 

What bits of information would you like to have which you don't already have? 
(List on Information Sheet, Question II. ) We realize you may not have all the 
information you would like for this flight, but for the purposes of the scenario, 
assume you are satisfied and we will proceed. 
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(•Remove Fig. I end replace with Fig. II) 


DESCRIPTION OF THE FLIGHT 

Use Fig. II to follow the progress of your flight. 

You were cleared to the Glens Falls airport "as filed". You lifted off 
from Bangor at lOrOv a.m. , and your departure was routine. At 10:14 
(14 minutes after departure) you reached your cruising level of 8000 feet 
and were established on V3 northeast of the Augusta VQR. At 10:34 
(34 minutes after departure) you crossed the Augusta VOR within one min- 
ute of your ETA. You proceed on V39 and cross Label, Limer, and Neets 
* 

intersections all very close to your ETA's. You have been in instrument 
conditions since departure but the flight has been smooth. At 11:21 ( 1 hour 
21 minutes after departure) you cross Grump intersection. One minute later 
you hear a short burst of static noise over your radio speakers. At the same 
time you notice your VOR needles and their "on-off" flags flicker unsteadily 
and return to normal indications. Curious to know what caused these events, 
you glance over the instrument panel and find a "zero" reading on the ammeter. 
You actuate the landing light and notice no change in ammeter indications. 

From this information you conclude the alternator has failed. You follow the 
procedures in the manual but your attempts to bring the alternator back into 
service are unsuccessful. Therefore, you turn off the alternator, minimize 
the electrical load, and operate solely on battery power. 

The battery, by itself, can supply the required power to operate your 
radios for only a limited time. The amount of time you have depends on the 
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size and condition of the battery, and the power requirements of the essential 
electrical equipment you use. Even under ideal conditions battery power 
is not expected to last longer than 50 minutes. 

. You are at an altitude of 8000 feet, just west of Grump intersection. TTie 
time is now 11:23 and you have been airborne for 1 hour and 23 minutes. 
Winds are out of the southwest at 30 knots. 


AlOl 


Clearly there is a need to divert from the planned flight 

and to land somewhere else. Glens Palls, your destination, is 

beyond your range in terms of battery time. Your tasX now is 

to find a suitable place to land. 

We will replace Figure II with Figure III which is a 

simplified version and includes all the airports in the area. 

(It should not be assumed that all these airports are within 

your range). I will act as the air traffic controller and 

provide you with information about the airports as you request 

it. The information which I am prepared to give you about 

PA use 

each airport is listed in Table IV. ^ You request each piece 
of information by stating the airport name (i.e. "C") and the 
particular information you desire about that airport (i.e. 
"bearing and distance"). You can ask for only one piece of 
information at a time. You will have two minutes to conduct 
your search and select an airport to divert to. You will be 
able to fly to that airport and shoot one approach only. 
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Ranking Exercise Instructions 


You have just finished choosing an airport to divert to in 
the face of a serious problem. Now we would like you to consider 
yourself to be in that same situation again. I have a set of 
cards here; each card describes an airp>ort in terms of ATC 
services, weather, the flight time from your present position to 
the airport, and the approach facilities there. We would like 
you to rank these airports from your "most preferable" to "least 
preferable," given the same situation. Recall that you have, at 
the very most, 50 minutes of battery time left. You may find it 
useful to divide the airports into "sub-groups," rank the airports 
in each sub-group, and then reconnect the sub-groups as appropriate 
Afterwards, make a final check of your complete rank and adjust 
it as you think necessary. 
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Go - No Go Instructions 


You have just finished ranking these airports from "most 
preferable" to "least preferable." The airport on the top of thi 
deck is the one you like the most, and the airport on the bottom 
is the one you like the least. Presumably, you would choose to 
land at this "top" airport first, in this situation, given these 
airports. 

Now assume that maintenance facilities to repair your air- 
plane are not available at the top airport. If you land where 
maintenance services are not available, you will probably be de- 
layed an extra day or so in order to make arrangements to have 
your aircraft repaired. If you knew airport "2" had maintenance 
facilities, would you "pass up" airport "1" for airport "2"? 

(If subject responds "Yes") Now assume maintenance services 
are not available at airport 2. If maintenance was available at 
airport 3, would you pass up airport 2 in favor of airport 37 

(If subject resp)onds "Yes" continue this routine until he 
says "No". Record the rank order (3rd letter of the code in 
lower right corner of card) and the Go - No Go point (by placing 
a slash in the rank between the "go" and "no go" points)). 
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TABLE I 


Important Specs, and Performance Figures 


Cruise Svx>ed 
Fuel Flow (65%pwr.) 
Usable Fuel Capacity 
Endurance 
Range 


135 KTAS (65% pwT. @ 7000 feet) 

10 GPH 
48 gallons 

4.8 hours (no reserve) 

648 nautical miles (no wind, no reserve) 


IFR Equipment on Board 

2 NAV/COMMs 
2 VOR/llf? Indicators 
1 ADF 

1 TTiree-light marker beacon receiver 
1 Transpomier (not encoding) 

1 Single axis autopilot 
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TABLE n 


for Glens Falls (New York): Hie weaUier is currently ”1000 
feet overcast and 3 miles visibility in rain. ” It is forecast 
to stay that way until 1:00 p.m. , local time, when it should 
improve to 1500 overcast and 5 miles visibility. 


for Bangor (Maine): Hie weather is currently ”1000 feet over- 
cast and 3 miles visibility in rain and fog. ” It is forecast to 
remain unchanged except for a chance of 500 feet overcast and 
1 mile visibility in rain, drizzle, and fog. 


for Albany (New York): The weadier is currently "1000 feet 
overcast and 4 miles visibility in light rain. ” It is forecast 
to remain the same until 1:00 p.m. , at which time it should 
improve to ”1500 overcast and 4 miles. ” 


Winds aloft: from the southwest (200*) at 30 knots at all 
altitudes up to 9000 feet. 


Icing Level: 10,000 feet. 


No PlREPs reported for the route. 
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LE nr. 


Form Approvod; 0MB No. 04>R0072 


T MANS^OAT ATiON 

#tOCAAC AVIATIO** A 0**»««l9T A A V lON 

FLIGHT PLAN 


!VIL .AIRCRAFT PILQTj . FARPamPI rrouirci you fUc an IFR Oighi plan tooporait under initnimciil 
ni|hi rulei in coniroUcdVirtpaet. Failurt to flit could reaull in a cm penalty not to eiceed SI .000 for each 
violation (Section 901 of the Federal Aviation Act of 19SI, at amended). Filing of a VFR f1i|ht plan ia 
recommended aa a good operating practice. See alto Pan 99 for requirementt concerning DVFR nigtii plan^ 
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TABLE IV 


ATC has the following information on each airport: 


- ATC Services : tower w/radar_(not GCA) , Jbower, FSS .or. none. 

- Visibility : Visibility in statute miles reported by an 

authorized weather observer. 

- Magnetic Bearing and Distance : From present position to 

the airport in nautical miles. 

- Ceiling : Height of lowest cloud cover (feet, AGL) reported 

by an authorized weather observer. 

- Terrain : Brief description of the topography of the land 

surrounding the airport (i.e. level, hilly, or 
Mountainous, etc.). 

- Approach Aids : The most accurate typ>e of approach at that 

airport (i.e. ILS , LOC, VOR, NDB, etc.). 
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Diagnosis Problem 

Subject # 

Experimenter 


DIAGNOSIS INFORMATION 


AIRFRAME 

□ wings & flaps - normal 

I ^ cowl - normal 

[ ^ windscreen - normal 

PANEL INSTRUMENTS 
. Attitude & Performance 

□ airspeed - N - 135 kts. 

I ) turn t bank - N - level 

I j art. horizon - N - level 

I j directional gyro - N - 040 

[ I vertical speed - N - 0 

[ '] altimeter - N - 7000' 

[ I mag. compass - N - 040 

I I stall warning - N - off 

[ ~] suction gauge - N 

Navigation t Communication 
I I transponde" - normal 

I I ornni 1 - normal 

I ] Omni 2 - normal 

I ] comm 1 - normal 

[ ] comm 2 - normal 

[ ] ADF - normal 


□ external noise - normal 

□ 'outside air temp. - normal 04 * F) 

□ cabin - oil drops on floor in front of pilot 


Engine Gauges 

□ tachometer - normal 

□ fuel - normal 

□ oil pressure - extremely low - near peg. 

□ oil temp. - normal 

□ cyl. head temp. - normal 
I I manifold press. - normal 

□ ammeter - normal 

I I exhst. gas temp. - normal 

□ fuel flow - normal 

Other 

I 1 panelllghts- p.rmal 

□ circuit breakers - normal 

□ annunciators - normal 


Information Not Listed 
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□ 

□ 

D 


COCKPIT CONTROLS 

loaifi 1 ruble. n i 1 

Subiect # J 


1 throttle - normal 

Experimenter ! 

I 1 master switch - on 1 


1 1 increase - normal R 

1 1 off - power lost 


1 ] decrease - normal R 

1 1 magneto L starter sw - both 

□ 

mixture - normal 

1 1 left - RPM drop 


1 1 increase - normal R 

I I right - RPM drop 


[ 1 decrease - normal R 

I I off - engine quits 

□ 

propeller RPM - normal 

I I fuel selector - left 


1 1 increase - normal R 

I I right - no change 


1 1 decrease - normal R 

I I left - no change 

□ 

gear selector - up 

I I off - engine quits 


1 1 gear down - normal R 

I I alternate static source- closed 

- 


I I open - no chanse 

□ 

yoke - trimmed for cruise 

P 1 control actions not listed 1 


1 1 pitch up/down - normal R 

1 I pitot heat - no change ' 

1 


1 1 turn left/righf - normal R 

□ 


1 1 pitch trim - normal R 

n 


NAVIGATION AND COMMUNICATION 

□ 

transponder - normal 

1 1 comm 1 - normal 


1 1 off - normal It 

1 1 off - normal R 


1 1 chk. CB - normal R 

1 1 chk. CB - normal R 


1 J change code - normal R 

1 ] comm 2 - normal 

□ 

omnl 1 - normal 

1 ] off - normal R 


f I off - normal R 

I 1 chk, CB - normal R 


1 1 chk. CB - normal R 

I 1 ADF - normal 

□ 

omnl 2 - normal 

! ] off - normal R 


1 1 off - normal R 

[ 1 chk. CB - normal R 


1 ] chk. CB - normal R 
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Diagnosis Problem 

Subject # 

Experimenter 


DIAGNOSIS INFORMATION 


AIRFRAME 

I I wings & flaps - N 

[ ^ cowl - N 

I I windscreen - N 

PANEL INSTRUMENTS 

Attitude & Performance 

I I airspeed - increased by 10 kts. to 
145 kts. 

[ I turn & bank - slight left bank 

I I art. horizon - level, no movement 

I ] directional gyro - no movement (250*) 

I I vertical speed - increasing negative 

[ I altimeter - decreasing slowly from 6000 

I I mag. compass - rotating slowly (230*) 

[ ^ stall warning - off 

I I suction gauge - extremely low, 

near peg 

Navigation & Communication 
I transponder - N 
omni 1 - N , 
omni 2 - N 
comm 1 - N 
comm 2 - N 
ADF - N, moving 


I I external noise - engine speed increasing 

□ ‘outside air temp. - N (37* f) 

□ cabin - N 

Engine Gauges 

I 1 tachometer - slight increase 

I I fuel - N 

□ oil pressure - N 

□ oil temp. - N 

I 1 cyl. head temp. - N 

□ manifold press. - N 

□ ammeter - N 

□ e.xhst. gas temp. - N 

□ fuel flow - N 

Other 

( ] panel lights - n 

% 

I I circuit breakers - N 

I 1 annunciators - N 

Information Not Listed 

□ 

n 

D 
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Diugnosis I I'ouic.n 


COCKPIT CONTROLS 


Subject # 
Experimenter 


throttle - 

N 


master switch - on 


n 

increase - NR 


1 1 off - power lost 


u 

decrease - NR 

□ 

magneto & starter sw - both 

□ 

mixture - 

■ N 


1 1 left - RPM drop 


□ 

increase - NR 


1 1 right - RPM drop 


□ 

decrease - NR 


1 1 off - engine quits 

□ 

propeller RPM - N 

□ 

fuel selector - left 


□ 

increase - NR 


1 1 right - no change 


[□ 

decrease - NR 


1 1 left - no change 

□ 

gear selector - up 


I 1 off - engine quits 


□ 

gear down - NR 

□ 

alternate static source- closed 

— 




1 1 open - no change 

□ 

yoke - slightly forward, slightly left 

□ 

control actions not listed 


□ 

pitch up/down - NR 


1 1 pitot heat - no change 


□ 

turn left/right - NR 


rn 


□ 

pitch trim - Ml 


n 



NAVIGATION AND COMMUNICATION 

□ 

transponder - ^ 

n 

comm 1 - N 


□ 

off - NR 


1 1 off - NR 


□ 

chk. CB - NR 


1 1 chk. CB - NR 


□ 

change code - NR 

□ 

comm 2 - N 

□ 

omni 1 - 

N 


1 1 off - NR 


□ 

off - NR 


1 1 chk. CB - Ml 


□ 

chk. CB - NR 

□ 

ADF - N 

□ 

omni 2 - 

N 


[ 1 off - NR 


□ 

off - NR 
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1 1 chk. CB - NR 


1 J 

chk. CB - NR 




Diagnosis Problem 

Subject # 

Experimenter 


DIAGNOSIS information 


AIRFRAME 


[ j wings Si flaps - N 

□ 

[ I cowl - N 

□ 

1 1 windscreen - N 

□ 


PANEL INSTRUMENTS 

Attitude & Performance 


□ 

airspeed - decrease of 10 kts. 

□ 

□ 

turn & bank - N - level 

□ 

□ 

art. horizon - N - level 

□ 

rn 

directional gyro - N - 360® 

□ 


vertical speed - N - 0 

□ 

LJ 

altimeter - N - 5000' 

□ 

rn 

mag. compass - N - 360® 

□ 

i_j 

stall warning - N - off 

□ 

□ 

suet ion gauge - N 

□ 

Navigation & Communication 


□ 

transponder - N 

□ 

□ 

Omni 1 - N . 

□ 

□ 

Omni 2 - N 

□ 

□ 

comm 1 - N 



comm 2 - N 

□ 

□ 

ADF - audio crackling, needle 

n 

wandering 


A 117 
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external noise - rough, vibrating engine 
with backfiring 
"outside air temp. - N - (40® F) 

cabin - being shaken 
Engine Gauges 

tachometer - RPM drop of 200 
fuel - N 

oil pressure - N 
oil temp. - N 

cyl. head temp. - slight decrease 
manifold press. - slight Increase 
ammeter - N 

exhst. gas temp. - large fluctuations ’ 
fuel flow - very slight decrease 
Other 

panel lights - n 

circuit breakers - N 
annunciators - n 
I nformation Not Listed 


COCKPIT CONTROLS 


Subiect # 
Experimenter 

□ 

throttle - 

N 

□ 

master switch - on 


□ 

increase - 


1 1 off - power lost 


□ 

decrease - NR 

□ 

magneto & starter sw - both 

□ 

mixture - 

N 


1 1 left - engine runs smooth 


□ 

increase - NR 


1 1 right - engine sputters, quits 


□ 

decrease - NR 


1 1 off - engine quits 

□ 

propeller RPM - N 

□ 

fuel selector - left 


□ 

increase - NR 


1 1 right - no change 


□ 

decrease - NR 


1 1 left - no change 

□ 

gear selector - up 


1 1 off - engine quits 


□ 

gear down - NR 

□ 

alternate static source - closed 

— 




1 1 open - no change 

□ 

yoke - N 


□ 

control actions not listed 


□ 

pitch up/d own - NR 


1 1 pitot heat - no change 


□ 

turn left/right - NR 


rn 


□ 

pitch trim - NR 


□ 



NAVIGATION AND COMMUNICATION 

□ 

transponder - N 


comm 1 - N 


□ 

off - NR 


1 1 off - NR 


□ 

chk. CB - NR 


1 1 chk. CB - nr 


O 

change code - NR 

□ 

comm 2 - N 

□ 

Omni 1 - 

N 


1 1 off- NR 


n 

off - NR 


1 1 chk. CB - nr 


LJ 

chk. CB - NR 

n 

ADF - audio crackling, needle wandering 

□ 

Omni 2 - 

N 


1 1 off- NR 


□ 

off - NR 


1 1 chk. CB - nr 


I I chk, CB - NR 
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Diagnosis Problem 

Subject # 

Experimenter 


DIAGNOSIS INFORMATION 


AIRFRAME 

[ I wings & flaps - N 

□ 

external noise - N 

1 1 cowl - N 

□ 

'outside air temp. - N (30® F) 

1 1 windscreen - n 

tzi 

cabin - N 

PANEL INSTRUMENTS 

Attitude & Performance 


Engine Gauges 

1 airspeed - slowly decreasing from 

n 

tachometer - N 

100 kts. 

1 j turn & bank - N - level 

□ 

fuel - N 

j 1 art. horizon - N - climb 

cn 

oil pressure - N 

1 j directional gyro - N - (300®) 

□ 

oil temp. - N 

1 1 vertical speed - sluggish, low 

n 

cyl. head temp. - N 

(100 ft. /min. ) 

1 1 altimeter - low - only 6300' 

□ 

manifold press. - N 

1 1 mag. compass - N - (300) 

LH 

ammeter - N 

[ j stall warning - N - off 

□ 

exhst. gas temp. - N 

1 1 suction gauge - N 

□ 

fuel flow - N 

Navigation & Communication 


Other 

1 ] transponder - n 

□ 

panel lights - n 

r 1 Omni 1 - N . 



circuit breakers - N 

1 1 omni 2 - N 

\ J 

LJ 

annunciators - N 


j comm 1 - N 


comm 2 - N 


I I ADF- N 


□ 

□ 


Information Not Listed 



CO CKPIT CONTROLS 
N 

increase - NR, no airspeed 
Increase 

decrease - NK, no airspeed 
increase 
N 

increase - NR 

decrease - nR 

] propeller RPM - N 

I I increase - NR 

I I decrease - NR 

] gear selector - up 


f ] throttle - 

□ 

□ 

I mixture - 

□ 





Diugnosis Irowtcin d 

Subject # ___________ 

Experimenter 

master switch - on 

□ off - N - power lost 
maeneto & starter sw - both 

left - N - RPM drop 
right - N - RPM drop 
off - N - engine quits 
fuel selector - left 
I I right - N - no change 

□ left - N - no change 

I 1 off - N - engine quits 


□ 

□ 

□ 


gear down - NR 


alternate static source - closed 


] yoke - N 


open - airspeed suddenly increases, 
speed reads correctly but sluggish 

□ control actions not listed 


□ 

vert. 


I I pitch up/down - up - airspeed 
decreases faster; down - airspeed stabilizes 
I I turn left/right - NR 

I I pitch trim - NR 


□ 

□ 

break airspeed rase - 

break VSI case - 

□ 

break altimeter case - 


S pitot heat - no change 
ATION 


airspeed 
returns to 
normal 


transponder - N 

1 1 comm 1 - N 

□ 

off- NR 

□ 

off - NR 

LJ 

chk. CB - NR 

n 

chk. CB - nr 

□ 

change code - NR 

1 1 comm 

2- N 

Omni 1 - 

N 

□ 

off - NR 

□ 

off - NR 

n 

chk. CB - NR 

n 

chk. CB - nr 

1 1 ADr- 

N 

Omni 2 - 

N 

□ 

off- NR 

□ 

off - NR 

A118B 1 1 

chk. CB - NR 


chk. CB - nr 


□ □□uu 


p/p DIAGNOSIS SUMMARY 


Diagnosis Problem 

Subject # 

Experimenter 


Potential causes mentioned and sequence location 


□ 


Best guess on final diagnosis 

Time of final diagnosis , or time out|] 

Is guess or diagnosis correct? | | yes | [ no 

Estimated time plane would fly 

Judged criticality (scale 1-7) 

After pilot knows correct cause of problem: 

estimated time plane would fly 

judged criticality (scale of 1-7) 
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Conditions for the Use of the Additive Model 
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Conditions For the Use of the Additive Model 


It has long been recognized in the conjoint measurement approach that 
situations could exist where numerical "goodness -of-f it" metrics would not 
be able to determine if the proposed composition rule is actually the best. 
Conjoint measurement addresses the issues of composition and measurement 
simultaneously. If, however, the dependent variable does not follow the stated 
composition rule to a suitable degree, it may be because either the composition 
rule is invalid, or the numerical scales are inadequate. At this point an 
analysis of the ordinal input data is required to determine which of the possible 
explanaticns holds. Krantz and Tversky (1971) outlined several testable 
properties of ordinal input data which could be used to determine which of the 
various composition rules describes the actual process. TTiey intended for 
their "axiomatic" approach to be considered in a complementary fashion with 
numerical goodness-of-fit approaches. 

In cases where any factor is varied over two levels, the property of 
ordinal independence is the only one required for the additive model to be 
considered as feasible. This is not to be confused with the property of 
orthogonality between attributes. An explanation of this independence property 
is given in the words of Krantz and Tversky: 

"The essence of ordinal independence is that the ordering of 
the dependent variable can be used to order some of the independ- 
ent variables (l.e. , factors) in a manner that does not depend on 
the remaining variables." (Krantz and Tversky, 1971). 

A mathematical representation of ordinal indei)endence can be developed in 
the following way. If the two levels for each of the four attributes are 
described as in Equation E2, 

Xj = a or b, 

X 2 " g or f, 

Xg = p or q, and 

x^ = yorz, (E2) 


then each alternative could be defined by the attribute values which are assigned 
to it (for example, (a,g,q,y) is the alternative where Xj = a, X 9 = g, X 3 = q, and 

x^ = y). If (a,g,p,y)2(b,g,p,y), and (a,f,p,y) ^(b,f,p,y) and, in general, if 
(a,X 2 ,x^,x^) 2 ;(b,X 2 »’^ 3 **_j) for all possible combinations of the values of X 2 , 

X 3 , x^, then Xj is said to be ordinally Independent of x^, X 3 , and x^. Independence 
of the other attributes is defined in the same fashion. 

One problem that must be considered when testing any axiom is the presence 
of error in the raw (ranked) data. In most cases, the data are regarded as 
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"fallible" In the sense that slight fluctuations or local errors occur In the 
ranks. This can lead to refuting an ordinal property on a strict basis, 
even though the property is largely satisfied. Unless the compliance 
criterion is relaxed to include those cases where the property is strongly 
though not strictly obeyed, no composition rule can be supported by these 
tests. 

In order to account for the effects of fallible data, Krantz and Tversky 
propose that ordinal Independence be expressed probabilistically to take 
on the view of statistical Independence. Required ordinal properties can 
be interpreted as statistical hypotheses and tested accordingly. This 
would minimize t^he effects of sli^t inconsistencies in the ranked data on 
the selection of a composition rule. 

A sensitivity analysis was performed to see how many inconsistencies 
could be introduced to the rank before the additive model became excessively 
distorted. The approach taken was to start with a rank which was in strict 
accordance with the Krantz-Tversky tests and progressively alter It. At 
each point the relative importance of each attribute was compared with that 
of the original rank (the relative importance of the attributes was determined 
from the coefficients obtained by the regression procedure outlined in 
Chapter 5). The initial rank for airports A through P was as follows: 

ABCDEFGHIJKLMNOP 

16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 


The numbers refer to the position each airport received In the order of pre- 
ference where sbcteen means "most preferable". Because the extremes in 
the preference positions (for example the positions of 16, 15, 2, or 1) have 
the most influence on the resulting coefficients, it was decided to move one 
of them around in a systematic fashion. The sixteenth position was progress- 
ively assigned to airports B,C,D,E and so on. After making the first 
alteration the rank appeared as follows: 

ABCDEFGH IJKLMNOP 

15 16 14 13 12 11 10 9 8 7 6 5 4 3 2 1 


The relative importance of the attributes was preserved and virtually 
100% of the variation was accounted for by the model. The second and third 
alterations were made as follows: 


ABCDEFGHIJK 
15 14 16 13 12 11 10 9 8 7 6 


L M N O P 

5 4 3 2 1 


A B 
15 14 


C D 
13 16 


E F 
12 11 


G H 
10 9 


I J 
8 7 


K L 
6 5 


M N 
4 3 


O P 
2 1 


A 122 


still the relative importance of the attributes was maintained, and the 
model accounted fo” rly 100^ of the response variation. It wasn't until 
airport G was ass'*’;.- . the sixteenth positio?- that the first change in relative 
importance of attributes was observed. 

ABCDEFGHIJKLMNOP 
15 14 13 12 11 10 16 9 8 7 6 5 4 3 2 1 

The additive model however still accounted for 95% of the variation. This 
analysis continued and, except for two more instances, the original relative 
Importance of attributes was maintained, and the model accounted for a good 
portion of the variation. 

This exercise is indicative of the ability of conjoint measurement and the 
additive model to absorb minor blemishes in the ranked data and still faith- 
fully reveal the subject's general ranking policy. It should be noted that the 
inconsistencies found in the actual data were all much less serious than those 
introduced in the sensitivity analysis. 
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APPENDIX J 


Master Summary of Data for the 
Destination Diversion Scenario 
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MASTER SUMMARY OF DATA 


Tho Master Summary of Data is given on the following 
page. The meaning of each category abbreviation is given 
below. 


BSUBATC 

is 

the 

value 

of 

the 

coefficient, B . . 

acc 

BSUBWX 

is 

the 

value 

of 

the 

coefficient, B 

wx 

BSUBTIM 

is 

the 

value 

of 

the 

coefficient, B. . . 

tim 

BSUBAPP 

is 

the 

value 

of 

the 

coefficient, B 

app 


ie:0WLEDG is the score received on the knowledge survey. 

TOTHRS is the total flying experience in hours. 

SEHRS is the total flight experience in single-engine 
airplanes , in hours. 

IFRHRS is total flight experience under Instrument 
Flight Rules, in hours. 

RATING is the grade of pilot certificate (PRIV» Private, 
COMM= Commercial, and ATP= Airline Transport). 

TRAINING is the typ>e of basic training received (CIV= 
Civilian, MIL« Military). 

FLYING is the type of flying most commonly done (GA/ 
COMM= General Aviation, Commercial). 

SPEAR is the value of the Spearman correlation co- 
efficient for the computed value-input rank pairs. 

MAINT is the number of airports the subject would "pass- 
up" in search for an airport with maintenance 
services . 

WOULDGO is the response the pilot gave to the question 
on wl ether he would attempt the flight or not. 

TOTALCOR is the "score "received on the four diagnosis 

scenarios conducted prior to the diversion-deci- 
sion scenario. 
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APPENDIX K 


Reduced Power Plagposls Interview 


The following interview was conducted with a widely known and respected 
aircraft mechanic. It demonstrates the ftaslbilitv of system problem diagnosis 
from verbal description of symptoms and the information seeking and problem 
solving logic used by an expert to narrow the list of plausible causes. 
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REDUCED POWER DIAGNOSIS 


P = Pilot (Brooks) C = Consultant (Kellenbarger) A = Auxilliary (Giffin) 

P Okay, I just noticed that I was cruising along at 2400 RPM, and my power started coming 

back — I've got an RPM loss. Do you have any idea what might be wrong? Or what we can 
do about it? 

C Have you tried carburetor heat? 

P No, ckay we can try that. No effect. 

C No effect. Do you have a drop; Can you pull heat on? 

P Yes 

C And you push it back off and does it raise back up to normal or above normal? 

P It comes back up to the reduced level to which it had dropped. To — below 24. 

C Oil pressure, oil temperature normal? 

P Well, let's take a look. Okay we notice the pressure appears to be normal. I've 

got a little advanced cylinder head temperature and oil temperature creeping up - it 
appears. 

C Is your mixture control full in? 

P Okay, we had leaned for cruise at altitude, so we can rlchen the mixture. And we get 

no significant improvement. 

C How about reducing the mixture control a little bit more than you had — what does that do? 

P Okay, we can try that and it doesn't seem to help. W'e have a very slow loss in RPM and 

the temperature gauges are both creeping up. 

C Have you switched tanks yet on your fuel tank? 

P Okay, we can try that, and we get no effect. 

C What happens if you lose a little altitude? Does this have any’thlng to do with your 


temperature ? 
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P Okay, we're on an IFR plan, so we're nailed at altitude, and we haven't talked to 

Center about this yet other than calling for help, so we could try to request a lower 
altitude. 

C Well, just a small amount to increase your air speed, what will that do? 

P Okay, \ seems that the effect of the power loss is advancing faster than the improvement 

we're getting from nossibly going down. 

A (At this point y . c beginning to be unable to hold altitude, too. ) 

P Right, that's going to happen real soon. 

C Okay, have 'mu played with the throttle in any amount — increase or decrease the throttle 

to see if that has any bearing on it? 

P Okay, we try fire-walling it and we get an increase In RPM's, but it's not going up to 

what it normally should go to. In other words, instead of getting maybe 2650, we get 
25 now "flat out". 

C I would say move the throttle back and forth abruptly, at least 5-10 hundred RPM's 

and see what happens and then leave it settle down. 

P Okay, we cycle throttle, and at this point, too, we've lost so much power that we're 

unable to hold altitude. And the throttle cycles — the RPM cycles, but it's still not 
going up to normal cruise. I'd say now we're down below 2000. 

A (2000 RPM, not 2000 feet?) 

P Right. 

C I'd say here you might switch mags and see what effect one mag has against the other 

one. See if you have one mag going bad and it's getting out of time for one reason or 
another. Be careful when you do this that you don't go over to a dead mag and leave it 
on too long . . . 
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P Lose everything? Okay, we switch mags, we get a little differential between them, but 

just as would be expected. We're definitely coming down now — 200 feet per minute. 

C Do you have any Indication of oil leaks on the cowling that you can recognize ? 

P Nothing apparent. Now the oil temperature and cylinder head temperature gauges are pegged. 

C I'd declare an emergency and proceed to the closest landing strip — I guess. 

P Okay, you wouldn't try to go over the mountains ? 

C Not losing altitude. 

P Do you think it would be better to try to put it down at on an off-airport landlng--try to 

go over and poke around in the soup at the airport with no tower and no reported weather 
or go back to the destination airport — that's 50 miles away, can we make It? It would 
take 30 minutes from here. 

C I think I'd head over to a destination that I'd know - knew it didn't have mountains in it. 

P So that means going back to Seaport ? 

C Yeah. 

P Okay, there's nothing else we can do in here? 

C Not unless it changes somewhere along the line in going back. \^Tien you change altitude 

it would hav6 something to do with it. 

P Okay, in the last minute or two that we've been talking, the RPV has stablized at somewhere 

between 1300 and 1400, and apparently with our loss in altitude, we've been able to go down 
to a point where we can stabllze here. It looks like the cylinder head and oil temperature 
gauges are still pegged. Our RPM loss is stablized here, we can hang on with this power 
level — and our airspeed is very low. Does this have any effect on the decision? 

C No, I would say that you ought to get to the ground as soon as possible or chang® altitude 

due to temperature — this may have some effect on what is wrong with your engine. 
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P Okay, we can’t go up at this point, we can only stay where we are or come down, 

so we may be able to hold 3000 feet to Seaport--just above stall. 

C Okay, I'd proceed to that destination. 

P Okay. 
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1. AGE: Age of the subject - categorized into intervals: 

1) age ^ 30 yrs. 

2) 30 yrs. <■ age < 50 yrs. 

3) age > 50 yrs. 

2. AIRPORTS: Airports the pilot was willing to pass to locate proper repair 

facilities. 

3. AP: Variable for airports used in computer runs valued (0) if airports < 2 

and (1) if airports > 2 . 

4. APP: Approach attribute of an airport. Includes ILS vs. NDB approach. 

5. ATC: Air Traffic Control attribute of an airport (presence of radar). 

6* B^pp: Pilots importance assessment of approach attribute of an airport. 

7* Pilots importance assessment of an air traffic control attribute of 

an airport. 

8. Pilots importance assessment of time. 

9. Pilots importance assessment of weather. 

10. Cl: Correctness score on Scenario #1 (possible correct; 0-5). 

11. C2: Correctness score on Scenario #2 (possible correct: 0-5). 

12. C3: Correctness score on Scenario #3 (possible correct: 0-5). 

13. C4; Correctness score on Scenario #4 (possible correct: 0-5). 

14. CAl: Subjective criticalit>’ estimate of event in Scenario #1 after being pro 

vided with the answer (scale 1-7; l=lowest criticality). 

15. CA2: Subjective criticality estimate of event in Scenario #2 after being pro 

vided with the answer (scale 1-7; l=lowest criticality). 

16. CA3: Subjective criticality estimate of event In Scenario #3 after being pro 

vided with the answer (scale 1-7; l=lowest criticality). 


A 133 



17. CA4: Subjective criticality estimate of event in Scenario #4 after being 

provided with the answer (scale 1-7; l=lowest criticality). 

18. CATSCRl: First category score on knowledge survey - knowledge sub- 

score for engine and fuel systems (possible correct: 0-7). 

19. CATSCR2: Second category score on knowledge survey - knowledge sub- 

score for electrical systems and cockpit instrumentation 
(possible correct: 0-7). 

20. C.ATSCR3: Third category score on knowledge survey - knowledge sub- 

score for weather and IFR operations (possible correct: 0-6). 


21. CBl: Subjective criticality estimate of event in Scenario *1 before being 

provided with the answer (scale 1-7: l=lowest criticality). 

22. CB.i: Subjective criticality estimate of event in Scenario ¥2 before being 

provided with the answer (scale 1-7; l=lowest criticality). 


23. CB3: Subjective criticality' estimate of event in Scenario #3 before being 

provided with the answer (scale 1-7; l=lowest criticality). 

24. CB4: Subjective criticality estimate of event in Scenario #4 before being 

provided with the answer (scale 1-7; l=lowest criticality'). 


25. C.nTRLI: Number of inquiries which involved control movements in 

Scenario #1. 

26. CNTRL2: Number of inquiries which involved control movements in 

Scenario #2. 


27. CNTRL3: Number of inquiries which involved control movements in 

Scenario #3. 

28. CNTRL4: Number of inquiries which involved control movements in 

Scenario #4. 


29. CNTRLTOT: Total number of inquiries for all four scenarios which 
involved control movements 

CNTRLTOT = CNTRLl ^ CNTRL2 CNTRL3 CNTRL4 


30. CORINQl: Ratio of correctness to inquiries for Scenario #1: 

CORINQl = Cl/ll 

31. CORINQ2: Ratio of correctness to inquiries for Scenario *2: 

CORINQ2 = C2/I2 
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32. C0RINQ3: Ratio of correctness to inquiries for Scenario #3: 

COR1NQ3 = C3/I3 

33. CORINQ4: Ratio of correctness to inquiries for Scenario #4: 

CORINQ4 = C4/I4 

34. CORINQT: Ratio of total correct to total inquiries for all four scenarios: 

CORINQT = (Cl C2 + C3 + C4)/ai ^ 12 + 13 + 14) 

35. DELTACl: Change in subjective criticality estimate of event for Scenario #1 

after being provided with the answer; DELTACl = CAl - CBl 

36. DELT.AC2: Change in subjective criticality estimate of event for Scenario #2 

after being provided with the answer; DELTAC2 = CA2 - CB2 

37. DELTAC3: Change in subjective criticality estimate of event for Scenario #3 

after being provided with the answer; DELTAC3 = CA3 - CB3 

38. DELTAC4: Change in subjective criticality estimate of event for Scenario *4 

after being provided with the answer; DELTAC4 = CA4 - CB4 

39. DIFl: Difference between number of total tracks and number of unique 

tracks in Scenario #1: DIFl = TTl - UTl 

40. DIF2; Difference between number of total tracks and number of unique 

tracks in Scenario #2: DIF2 = TT2 - UT2 

41. DIFS: Difference between number of total tracks and number of unique 

tracks in Scenario #3: DIFS = TT3 - UTS 

42. DIF4: Difference between number of total tracks and number of unique 

tracks in Scenario #4: DIF4 = TT4 - UT4 

43. DIFT: Difference between number of total tracks and number of unique 

tracks in all four scenarios: DIFT = TOTTRAKS - TOTUTRKS 

44. El: Efficiency score on Scenario #1: El = [25 - 2 (minutes to diagnose) - (II -2)j 

45. E2: Efficiency score on Scenario #2: E2 = [25 - 2 (minutes to diagnose) - (12 -2)] 

46. E3: Efficiency score on Scenario #3: E3 = [25 - 2 (minutes to diagnose) - (13 -2)1 

47. E4: Efficiency score on Scenario #4: E4 = [25 - 2 (minutes to diagnose) - (14 -2)] 

48. FLY: Computer variable for the variable flying; takes values: 

(0) if flying = 1,2,3, or 4 = non-pleasure 

(1) if flying = 5 = pleasure 
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58. 
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61. 
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FLYING: Most frequent kind of flying. 
Valued: (1) Airline 

(2) Commercial 

(3) Business 

(4) Military 

(5) Pleasure 


GAT: Participation in general aviation simulation; 0 = did not participate, 

1 = did participate 

GATKl: Open ended knowledge test on GAT subjects - subscore on engine 
operations (possible correct: 0-7). 

GATK2: Open ended knowledge test on G.AT subjects - subscore on fuel 
systems (possible correct: 0-7). 

GATK3: Open ended knowledge test on GAT subjects - subscore on electrical 
systems (possible correct: 0-7). 

GATK4: Open ended knowledge test on GAT subjects - subscore on cockpit 
Instrumentation (possible correct: 0-7). 

GATK5: Open ended knowledge test on GAT subjects - subscore on weather 
(possible correct: 0-7). 

GATK6: Open ended knowledge test on GAT subjects - subscore on IFR 
procedure (possible correct: 0-7). 


GATKT: Average of all parts of open ended knowledge GAT test: 

GATKT = GATKl + GATK2 ^ GATK3 GATK4 GATK5 GATK6 

6 


GONOGO: Designates whether the pilot would have taken the flight under the 
given conditions. Valued: (0) - would not go, (1) - would go. 

II: Number of Inquiries in Scenario #1. 

12: Number of inquiries in Scenario #2. 

13: Number of inquiries in Scenario #3. 

14: Number of inquiries in Scenario #4. 

IFR: Variable designating upper and lower quartiles of IFR hours- 

(0) if IFR hrs. * 175 

(1) if IFR hrs. ^700. 
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64. IFRHRS: Hours of flying under Instrument flight rules. 

65. INPTRl: Ratio of inquiries to total tracks in Scenario #1: INPTRl = Il/TTl. 

66. INPTR2: Ratio of inquiries to total tracks in Scenario #2: INPTR2 = I2/TT2. 

67. INPTR3: Ratio of inquiries to total tracks in Scenario #3: INPTR3 = I3/TT3. 

68. INPTR4: Ratio of inquiries to total tracks in Scenario #4: INPTR4 = 14/ TT4. 

69. INPTRT: Ratio of total inquiries to total tracks for all four scenarios: 

INPTRT = TOTINQ/TOTTR.AKS 

70. KNOW: Vari:ible designating upper and lower quartiles of KNOWLEDG scores 

(0) if KNOW’.EDG ^ 9 

(1) if KNOWLEDG 5= 16 

71. KNOWLEDG: Score on aircraft systems survey (possible correct: 0-20). 

72. LATELY: Relative amount of flying done in last year: 

(0) if pilot has more than 50 hours 

(1) if pilot has less than 20 hours 

73. Ml: Merit score on Scenario #1: Ml = (Cl) x (El). 

74. M2: Merit score on Scenario #2: M2 = (C2) x (E2). 

75. M3: Merit scor' on Scenario #3: M3 = (C3) x (E3). 

76. M4: Merit score on Scenario #4: M4 = (C4) x (E4). 

77. MECH: Mechanic: (0) = not a mechanic, (1) = mechanic. 

78. PROPCONl: Proportion of control movements to inquiries in Scenario #1: 

PROPCONl = CNTRLl/ll 

79. PROPCON2: Proportion of control movements to inquiries in Scenario *2: 

PROPCON2 = CNTRL2/'12 

80. PROPCON3: Proportion of control movements to inquiries in Scenario *3: 

PROPCON3 = CNTRL3/13 

81. PROPCON4: Proportion of control movements to inquiries In Scena-io #4: 

PROPCON4 = CNTRL4/I4 

82. PROPCONT: Proportion of total control movements to total inquiries in all 

four scenarios: PROPCONT “ CNTRLTOT/TOTINQ 
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83. RAT: Substitute variable for RATING used to plot initial data tables. Takes 

on same values as RATING. 

84. RATING; Rating type - 

1 = Private 

2 * Commercial 

3 * Air Transport 

85. RATSCORE: Variable dividing ratings into two groups - 

0 if private pilots (RATING * 1) 

1 if commercial or air transport pilot (RATING * 2 or 3) 

86. RECENCY: Relative amount of flying time in past year - 

1 = more than 50 hours 

2 = between 20 and 50 hours 

3 = less than 20 hours 

87. S: Specific subjects involved in the GAT experiment - 

0 for subject numbers 11, 31, 32, 33 

1 for subject numbers 28, 34, 35, 38 

88. SEHRS: Hours of flying in a single engine aircraft. 

69. SEHRSLOG; Natural logorithm of single engine flying hours; 

SEHRSLOG = LOGg (SEHRS) 

90. SHRSRANK: Variable designating upper and lower quartiles for single 

engine hours; 

0 if SEHRS 5 488.75 

1 if SEHRS 22075.25 

91. SUB: V'ariable dividing subjects - 

0 if subjec* number is ^ 30 

1 if subject number is > 30 

92. SUBJECT: Subject number (N = 40) 

93. T: Variable designating upper and lower divisions for the variable TIM: 

0 if TIM < .625 

1 if TIM > 1 

94. TC: Variable designating upper and lower quartiles of TOTCOR: 

0 if TOTCOR < 10 

1 if TOTCOR 2 17 

95. TDELTAC: Sum of the changes in subjective criticalitv estimates for all 

four scenarios; TDELTAC “ TCRITAFT - TCRITBEF 
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96. TE: Variable designating the upper and lower fjuartlles of TOTEFF; 

0 if TOTEFF < 42 

1 If TOTEFF ^59 

97. THRSLOG: Natural logarithm of totaly flying hours; 

THRSLOG “ LOGg (TOTHRS) 

98. THRSR.ANK: Variable designating upper and lower quartlles for total 

flying hours: 

0 if TOTHRS s 1007 

1 if TOTHRS a 5375 

99. TIM: Time attribute of an alternate airport - flying time to the airport 

100. TM: Variable designating upper and lower quartlles for total merit: 

0 if total merit s 129.25 

1 if total merit ^ 235 

101. TOTCOR: Total correct score for all four scenarios: TOTCOR “ Cl C2 C3 C4 

(possible cori^ect * 0-20). 

102. TOTCRITAFT: Total of subjective criticality estimates for all four scenarios 

after being provided with the answers; 

TCRITAFT » CAl + CA2 ^ CA3 * CA4 

103. TCRITBEF: Total of subjective criticality estimates for all four scenarios 

before being provided with the answers: 

TCRITBEF = CBl CB2 » CB3 + CB4 

104. TOTEFF: Total efficiency score for all four scenarios: 

TOTEFF * El - E2 * E3 E4 

105. TOTHRS: Total flying hours. 

106. TOTINQ: Total number of inquiries for all four scenarios: 

TOTINQ * II * 12 + 13 14 

107. TOT^tERIT: Total merit score for all four scenarios; 

TOTMERIT Ml * M2 - M3 M4 

108. TOTTR-AKS: Total number of tracks for all four scenarios; 

TOTTR.M'IS - TTl TT2 TT3 * TT4 

109. TOTUTRKS: Total number of unique tracks for all four scenarios: 

TOTUTRKS - UTl UT2 ♦ UT3 * UT4 
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110. TRA: Variable used to plot the TRAINING values in the data tables: 

1 ” military 

2 • civilian 

111. TRAINING; Type of training (military or civilian). 

112. TTl: Total number of tracks In Scenario #1. 

113. TT2: Total number of tracks in Scenario <*2. 

114. TT3: Total number of tracks in Scenario *3. 

115. TT4: Total number of tracks in Scenario *4. 

116. UTl: Number of unique tracks in Scenario #1. 

117. UT2: Number of unique tracks I.: Scenario *2. 

118. UT3: Number of unique tracks in Scenario *3. 

119. UT4: Number of unique tracks in Scenario *4. 

120. \VX; Weather attribute of an alternate airport; includes ceilings and visibilities. 

121. YOUNGOLD: Variable designating the upper and lo\\er divisions of the age 

category: 

0 if age <- 30 

1 if age > 50 

122. 7.1: Ratio of correctness to total tracks for Scenario #1; 

Z1 • Cl/TTl. 

123. Z2: Ratio of correctness to total tracks for Scenario #2; 

Z2 » C2/TT2. 

124. Z3: Ratio of correctness to totrl tracks for Scenario *3: 

Z3 * C3/TT3. 

125. 74 ; Katio of correctness to total tracks for Scenario »4: 

Z4 • C4 TT4. 

126. 7T: Ratio of total correct to total number of tracks for all four scenarios: 

ZT • (Cl ♦ C2 ♦ C3 * C4)/(TT1 ♦ TT2 - TT3 ♦ TT4) 


